Automatic detection of user&#39;s periods of sleep and sleep stage

ABSTRACT

Aspects of automatically detecting periods of sleep of a user of a wearable electronic device are discussed herein. For example, in one aspect, an embodiment may obtain a set of features for periods of time from motion data obtained from a set of one or more motion sensors in the wearable electronic device or data derived therefrom. The wearable electronic device may then classify the periods of time into one of a plurality of statuses of the user based on the set of features determined for the periods of time, where the statuses are indicative of relative degree of movement of the user. The wearable electronic device may also derive blocks of time each covering one or more of the periods of time during which the user is in one of a plurality of states, wherein the states include an awake state and an asleep state.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/859,192, filed Sep. 18, 2015, which is hereby incorporated byreference in its entirety. This application claims the benefit of U.S.Provisional Application No. 62/054,380 filed Sep. 23, 2014, which ishereby incorporated by reference in its entirety; U.S. ProvisionalApplication No. 62/063,941 filed Oct. 14, 2014, which is herebyincorporated by reference in its entirety; U.S. Provisional ApplicationNo. 62/067,914 filed Oct. 23, 2014, which is hereby incorporated byreference in its entirety; and U.S. Provisional Application No.62/068,622 filed Oct. 24, 2014, which is hereby incorporated byreference in its entirety.

FIELD

The embodiments are related to the field of wearable electronic devices.Specifically, the embodiments are related to automatic movementdetection utilizing a wearable electronic device.

BACKGROUND

Wearable electronic devices have gained popularity among consumers. Awearable electronic device may track user's activities using a varietyof sensors and help the user to maintain a healthy life style. In orderto determine a user's activities, a wearable electronic device collectsactivity data and runs computations on that data. One difficulty ofobtaining accurate determinations of a user's activities is that thesewearable electronic devices, because they are worn by a user, aretypically packaged in a compact casing containing less powerfulprocessor(s) (on which it is harder to run complex computations) thanlarger electronic devices.

Many wearable electronic devices may track metrics related to particularactivities, such as a step count metric for running and walkingactivities. Other metrics that may be tracked by a wearable electronicdevice include metrics related to sleep. Typically, to initiate trackingof sleep metrics, a wearable electronic device may contain an interfacefor the user to provide notification that the user plans ontransitioning into the asleep state (e.g., through the user pushing abutton of or tap on the wearable electronic device).

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments described herein are illustrated by way of example, andnot by way of limitation, in the figures of the accompanying drawings inwhich like references indicate similar elements.

FIG. 1 illustrates user state detection and user sleep stage detectionutilizing movement measures according to one embodiment of theinvention.

FIG. 2A illustrates movement measures collected at different timewindows for moments of interest, as well as determining statisticalfeatures for one of the moments of interest and classifying that momentof interest based on the statistical features, according to oneembodiment of the invention.

FIG. 2B illustrates classifying user active/inactive status and furtherderiving awake/asleep state based on the movement measure according toone embodiment of the invention.

FIG. 3A illustrates movement measure snapshots taken from data duringdifferent time spans according to one embodiment of the invention.

FIG. 3B illustrates the determination of a statistical feature formoments of interest using data during different time spans according toone embodiment of the invention.

FIG. 3C illustrates the classifying user active/inactive status atmoments of interest at the live system during different time spansaccording to one embodiment of the invention.

FIG. 3D illustrates the deriving user awake/asleep state using dataduring different time spans according to one embodiment of theinvention.

FIG. 3E illustrates the deriving user active/inactive/not-worn status atthe live system during different time spans according to one embodimentof the invention where not-worn status is included.

FIG. 3F illustrates the deriving of user's user awake/asleep state usingdata during different time spans according to one embodiment of theinvention where not-worn status is considered.

FIG. 4A illustrates the determining statistical features at differenttime windows for moments of interest, as well as determining statisticalfeatures for one of the moments of interest and classifying the momentof interest based on the statistical features according one embodimentof the invention.

FIG. 4B illustrates the result of the determination of sleep stages forthe moments of interest at different time windows according oneembodiment of the invention.

FIG. 5 is a flow diagram illustrating movement measure generation, andoptional user state detection and/or user sleep stage detection,according to one embodiment of the invention.

FIG. 6 is a flow diagram illustrating automatic detection of user'speriods of sleep according to one embodiment of the invention.

FIG. 7 is a flow diagram illustrating automatic detection of user'ssleep stages according to one embodiment of the invention.

FIG. 8A is a flow diagram illustrating automatically reducing powerconsumption of at least one of a photoplethysmographic sensor and amotion sensor according to one embodiment of the invention.

FIG. 8B is a flow diagram illustrating automatically increasing powerconsumption of at least one of a photoplethysmographic sensor and amotion sensor according to one embodiment of the invention.

FIG. 9 is a block diagrams illustrating a wearable electronic device andan electronic device implementing operations disclosed according to oneembodiment of the invention.

FIG. 10A illustrates a wearable electronic device placed on its side ona flat surface when not being worn according to one embodiment of theinvention.

FIG. 10B illustrates a wearable electronic device placed on its face ona flat surface when not being worn according to one embodiment of theinvention.

FIG. 10C illustrates a wearable electronic device placed on a flatsurface such it is perpendicular to that flat surface when not beingworn according to one embodiment of the invention.

FIG. 10D illustrates a wearable electronic device being placed on itsback on a flat surface when not being worn according to one embodimentof the invention.

FIG. 10E illustrates orientations of a wearable electronic device when auser is engaged in various activities according to one embodiment of theinvention.

FIG. 11 is a flow diagram illustrating the automatic detection of when awearable electronic device is not being worn based on an accelerometeraccording to one embodiment of the invention.

FIG. 12 illustrates exemplary alternative embodiments for implementingblock 1104 from FIG. 11.

FIG. 13 illustrate operations relating to not-worn state detectionutilizing accelerometer measures according to one embodiment of theinvention.

FIG. 14A illustrates the recordation of consecutive period of time aseither a worn state or a not-worn state based on acceleration datameasured along an axis of an accelerometer exceeding a thresholdacceleration for that period of time according to one embodiment of theinvention.

FIG. 14B illustrates the derivation of spans of time when a wearableelectronic device is not being worn based on the states recorded forconsecutive periods of time according to one embodiment of theinvention.

FIG. 15A illustrates detection of a span of time being in a not-wornstate for a first axis according to one embodiment of the invention.

FIG. 15B illustrates detection of a span of time being in a not-wornstate for a second axis according to one embodiment of the invention.

FIG. 16 is a block diagram illustrating the wearable electronic deviceand an electronic device implementing operations disclosed according toone embodiment of the invention.

DETAILED DESCRIPTION

Embodiments described herein may refer to operations and features of awearable electronic device. A wearable electronic device may beconfigured to measure or otherwise detect motion experienced by thewearable electronic device. Such motion, for simplicity of discussion,may be described with reference to a display of the wearable electronicdevice, where the display is generally located on the user's forearm inthe same place the display of a wrist watch would be located. Whileembodiments can be described with reference to a display of the wearableelectronic device being generally located on the user's forearm in thesame place the display of a wrist watch would be located, the scope ofthe invention is not so limited because modifications to a wearableelectronic device can be made so that the wearable electronic device canbe worn on a different location on the body (e.g., higher on theforearm, on the opposite side of the forearm, on a leg, on a torso, aseye glasses, and so forth) will be apparent to one of ordinary skill inthe art.

Some example embodiments may involve a wearable electronic device thatgenerates movement measures based on movement data generated by sensorsof a wearable electronic device. A “movement measure,” as used herein,may refer to data or logic that quantifies multiple samples of motiondata generated by motion sensors of a wearable electronic device. Inmany cases, the movement measure will consume less memory to store thanthe aggregate of the multiple samples of motion data that the movementmeasure quantifies. In the case of accelerometer data, a movementmeasure may quantify acceleration data across multiple samples from theaccelerometer along one or more axes. For example, a movement measuremay be calculated every 30 seconds, where the movement measurequantifies the samples of accelerometer data generated for those 30seconds. Further, the quantification can be a single number indicativeof the degree of motion experienced across multiple axes for themultiple samples.

Although a movement measure can be used to decrease the memory consumedby storing multiple samples of movement data, it is to be appreciatedthat a movement measure is different than simply compressing themultiple samples of motion data. Such is the case because the compressedmotion data generally would need to undergo decompression before beinguseful. That is, the compressed data itself, without decompression,generally does not represent a degree of movement for the time periodsin which the motion data was sampled. In comparison, a value of amovement measure by itself can represent some notion of a degree ofmovement experienced during the timer intervals corresponding to thesamples of motion data. In this way, one movement measure may becompared against another movement measure to determine a difference inthe degree of motion experienced during different time intervals.

In an example embodiment, a wearable electronic device that is to beworn by a user may include a set of one or more motion sensors togenerate motion data samples that represent motion of the wearableelectronic device. The motion data samples include a first set of one ormore motion data samples generated during a first time interval and asecond set of one or more motion data samples generated during a secondtime interval. The wearable electronic device may be configured to(e.g., by a set of one or more processors executing instructions) obtainthe motion data samples generated by the set of motion sensors. Thewearable electronic device may then generate a first movement measurebased on a quantification of the first set of motion data samples. Thewearable electronic device may also generate a second movement measurebased on a quantification of the second set of the motion data samples.The wearable electronic device may store the first movement measure andthe second movement measure as time series data in a machine-readablestorage medium.

Alternatively or additionally, some embodiments may automatically detectblocks of time where a wearer of the wearable electronic device isasleep. Here, automatic detection may refer to a detection that occurswithout an explicit instruction from the wearer, such as traversingon-screen menus or otherwise inputting periods of time where the weareris sleeping. It is to be appreciated that the term “automatic” does notpreclude a case where the user may enable such a feature or act in a waythat does not represent an explicit instruction of the timing when auser transitions between different sleep states. Such an embodiment thatdetects blocks of time in which the wearer is asleep may obtain a set offeatures for one or more periods of time from motion data obtained froma set of one or more motion sensors or data derived therefrom. Examplesof data derived from the motion data may include the movement measuresdescribed above. An embodiment may then classify the one or more periodsof time as one of a plurality of statuses of the user based on the setof features determined for the one or more periods of time. The statusesare indicative of relative degree of movement of the user. In some casesthe statuses may be enumerated state values, such as active or inactive,where each state represent a different degree of movement. In othercases, the statuses may be numerical values or range of values thatquantify the degree of movement. The embodiment may then derive blocksof time covering the one or more periods of time during which the useris in one of a plurality of states. The states include an awake stateand an asleep state.

In some cases, some embodiments may operate to adjust the powerconsumption of a sensor based on the state of the user. For example,based on detecting that a state of the user, as tracked by a wearableelectronic device, has transitioned into an asleep state, an embodimentmay decrease power consumption of at least one sensor. Further, based ondetecting that the state of the user, has transitioned out of the asleepstate, an embodiment may reverse the decrease of power consumption ofthe at least one sensor.

In some cases, some embodiments may operate to detect when in time thewearable electronic device is not being worn by the user. Someembodiments that detect when the wearable electronic device is not beingworn may automatically determine a period of time when the wearableelectronic device is not being worn based on a comparison of motion dataobtained from a set of motion sensors and a not-worn profile. Thenot-worn profile may be data or logic that specifies a pattern of motiondata that is indicative of when the wearable electronic device is notworn by the user. The embodiment may then store, in non-transitorymachine readable storage medium, data associating the period of timewith a not-worn state.

Example embodiments are now discussed with reference to the Figures.

FIG. 1 illustrates user state detection and user sleep stage detectionutilizing movement measures according to one embodiment of theinvention. The task boxes and blocks of FIG. 1 may be implemented in awearable electronic device, or distributed between the wearableelectronic device and one or more other electronic devices coupled tothe wearable electronic device. Electronic devices coupled to thewearable electronic device may be referred to herein as a secondaryelectronic device. A secondary electronic device may refer to a server(including hardware and software), a tablet, a smartphone (possiblyexecuting an application (referred to as an app)), a desktop, or thelike. A secondary electronic device can implement, for example, blocks122/190/192. Additionally, or alternatively, a secondary device thatprovides sensor data can implement, for example, blocks 122 and/or 114.Task boxes 1-5 illustrate an example order in which operations may beperformed by the components shown in FIG. 1. It is to be understood,however, that other embodiments may perform task boxes 1-5 in an orderthat differs from that shown in FIG. 1.

At task box 1, a motion sensor (e.g., a multi-axis accelerometer) 112generates motion data samples that represent motion for a plurality oftime intervals (e.g., the motion sensor 112 may be in a wearableelectronic device and the motion data represent motion of that device).The motion sensor may generate a number of motion data samples during atime interval. The number of samples generated in a time interval maydepend on the sampling rate of the motion sensor and the length of timeof the interval. In the case that the motion sensor is an accelerometer,the motion data samples may characterize a measurement of accelerationalong an axis of movement. In some cases, a motion data sample may be adata value that consumes a given amount of storage (e.g., 64-bits,32-bits, 16-bits, and so on). The amount of storage consumed by a samplemay depend on the implementation of the accelerometer used in theembodiment and the communication interface used to transfer the samplesfrom the motion sensors to memory accessible by the microprocessor ofthe wearable electronic device.

At task box 2, the movement measure generator 116 generates a pluralityof movement measures based on the motion data samples. A movementmeasure may be used to reduce the storage space needed to store datathat characterizes movement of the wearable electronic device over time.For example, in one case, a movement measure can quantify of a number ofmotion data samples generated for a given time period (e.g., over a 30second time period) with a single value. Further, according to someembodiments, an individual movement measure may consume less storagespace than a single motion data sample. For example, a movement measuremay consume 4-bits, 8-bits, 16-bits, 32-bits, or any other storage size.

As is discussed in greater detail herein, there are a number oftechniques for generating movement measures. For example, in oneembodiment, the movement measure generator 116 may process movement datasamples at the moment that the sensor device generates the sample or,additionally or alternatively, may process a block of motion datasamples (e.g., samples for one or more time intervals) at apredetermined frequency or based on a number of samples.

The generation of movement measures can be used to determine userasleep/awake state, thus the movement measures can also referred to assleep coefficients for those embodiments that determine sleep states.There are at least two base versions of the sleep detection method. Thefirst is a motion sensor (e.g., an accelerometer) solution, which usesdata from a three-axis motion sensor. The second uses a combination ofdata coming from a motion sensor and an optical heart rate monitor (HRM)module including a photoplethysmographic sensor 114.

At task box 3, the statistical measurer 122 may analyze the movementmeasures to derive a time series of values for one or more statisticalfeatures that characterize a user's movement. In some cases, the featurevalue for a given movement measure may depend on the movement measuresnear (e.g., as may be measured by proximity in a time series) the givenmovement measure. For example, some embodiments of the statisticalmeasurer 122 may utilize a rolling window of movement measures togenerate a feature value for a given movement measure. To illustrate,assuming the set of movement measures is represented as MM={mm₀, mm₁, .. . , mm_(n), mm_(n+1), . . . mm_(z)}, then the statistical measurer 122may derive the feature value of mm_(n) from the movement measuresmm_(n−w−1) to mm_(n+w−1), where the window size is 2*w. Examples ofstatistical features that the statistical measurer can derive include,among others: a quantile (e.g., quartile of acceleration data), aninter-quantile range, measures of dispersion (e.g., Gini coefficients),measures of entropy, information in the frequency domain (e.g., aquantification of an amount of periodic motion and amount of periodicmotion in two or more frequency bands). Other examples of features arediscussed below.

At optional task box 1O, optionally a photoplethysmographic sensor 114may be utilized to generate photoplethysmography (PPG) data to calculateheart rate (HR), heart rate variability (HRV), and/or respiration rate(RR). A photoplethysmographic sensor typically includes a light sourceand a photodetector. A common light source is a light-emitting diode(LED).

At task box 4A, once the values for the set of feature have beenderived, the user activity classifier 124 may classify or otherwiselabel the periods of time corresponding with the statistical featureswith an activity level, such as an active level or an inactive level. Insome cases, the user activity classifier 124 classifies or otherwiselabels each statistical feature in the set of feature values, while, inother cases, the user activity classifier 124 may classify or otherwiselabel a subset of features in the time series for the features, such asevery other feature value, every five feature values, every ten featurevalues, and so on.

At task box 4B, the time block classifier 126 uses the activity levelsto assign blocks of time a sleep state selected from one of a pluralityof sleep states. The sleep states can include, among other things, anawake state and an asleep state. As described in greater detail below,some embodiments may include sleep states that represent differentstages of sleep of a user. Further, some embodiments may include sleepstates that represent different types of non-sleeping activities (e.g.,a restless state). Each block of time can span one or more of theperiods of time characterized by the activity levels. In some cases, theactivity levels covered by the blocks of time may be homogeneousactivity levels, while other cases may allow for heterogeneous activitylevels.

Thus, FIG. 1 illustrates a system that can efficiently and/orautomatically determine when a wearer of a wearable electronic device issleeping. For example, rather than storing all of the motion dataobtained from a motion sensor and deriving sleep states directly fromthe raw motion data, the system may instead transform the multiplemotion data samples for a time interval into a single numerical valuerepresented by the movement measure. Further, in some cases, an activitylevel for a time period of time may be determined from multiple featuresderived from the movement measures of multiple time intervals.

The operations and components of FIG. 1 are now described in greaterdetail.

Movement Measures

As just described above, embodiments may use a wearable electronicdevice containing a motion sensor to measure motion data (e.g.,acceleration data measured by an accelerometer) that are indicative ofmotion that the wearable electronic device has undergone. Also describedherein, a movement measure may quantify. In some embodiments, eachmovement measure is a single numeric number generated based on acombination (e.g., addition, average, and the like) of a quantificationof a distribution of the motion data along one or more axes of themotion sensor. The single numeric number is a compact representation ofthe user's motion, and, as described in the foregoing, it can be used todetect user's state, user's sleep stage, an activity level, and othertypes of status for the user.

Using the movement measure, quantified at some interval or frequency(e.g., each 30 seconds), embodiments create informative statisticalfeatures that characterize a user's movements over longer periods oftime (e.g., 10 minutes). In one embodiment, the movement measuresquantify user movements and behavior across a short interval of time(e.g., 30 seconds). More specifically, this metric quantifies thedistributions of acceleration data (a type of motion data) along axesand combines them into a single numeric number. Let a_x, a_y, a_zrepresent time series arrays of accelerometer data measured along threeaxes sampled at some sampling rate (e.g., 20 Hz) over some interval oftime (e.g., 30 seconds).

One embodiment of the movement measure uses the maximum and minimumacceleration along an axis to quantify the distribution of accelerationalong that axis. The separate values from each axis are summed togetherinto a single metric for each 30 second interval (other intervals arepossible as well). The movement measure generated from a set of motiondata generated over a time period may then be expressed as:MM=max(a_x)−min(a_x)+max(a_y)−min(a_y)+max(a_z)−min(a_z)

A more general form of this metric allows for different weighting alongdifferent axes and different exponential powers for each axis. w_x, w_y,w_z are weighting factors and exp_x, exp_y, exp_z are exponents. Themovement measure may be expressed as:MM=w_x*(max(a_x)−min(a_x))^exp_x+w_y*(max(a_y)−min(a_y))^exp_y+w_z*(max(a_z)−min(a_z))^exp_z

It is to be appreciated that some embodiments may perform a number ofother post-processing operations on a movement measure. For example, inother embodiments, a movement measure generator may perform operationswith saturation (the calculated movement measures are clamped to theboundary values supported by a number of bits used to store thosecalculated values). In some embodiments, a movement measure generatormay cast the calculated movement measure into a 4 bit variable (or 8bits, 16 bits), which may, in some cases, minimize data volume. In someembodiments, the frequency of occurrences of the calculated movementmeasures being saturated may be utilized to determine a threshold abovewhich the casting is shifted to generate the movement measures (e.g., toaccount for “noisier” motion sensors which cause the meaningful data tobe in higher order bits of the calculated movement measures). Forexample, in an embodiment that uses 4 bits to represent the movementmeasures, a shift of zero means the casting of the movement measures isfrom bit positions 3:0 of the calculated movement measures, a shift of 1means the casting of the movement measures is from bit positions 4:1 ofthe calculated movement measures, and so on.

In one embodiment, a calculated movement measure comprises a combinationof a statistical measure of the motion data along one or more axes. Thestatistical measure of the motion data may be one or more of a quantile(e.g., quartile of acceleration data), an inter-quantile range, measuresof dispersion (e.g., Gini coefficients), and measures of entropy. Inaddition, the statistical measures can use information in the frequencydomain in one embodiment. The information in the frequency domain may beused to quantify an amount of periodic motion and compare the amount ofperiodic motion in two or more frequency bands.

Other methods of quantifying the distributions of motion data, includingaccelerometer data, are also possible:

-   -   Using the standard deviation rather than the maximum minus the        minimum    -   Using the interquartile range of motion data.

Other methods for quantifying user movements are also possible, such as:

-   -   Using the temporal derivative (also known as jerk, surge, or        lurch) of motion data along each of the three axes.    -   Measuring deviations of total acceleration from 1 G (the        gravitation acceleration at sea-level on Earth).    -   Using integrals of the accelerometer data (e.g., first integral        is velocity, second integral is position).

Note although acceleration data measured by an accelerometer is given asan example of motion data, other sensors (e.g., a gyroscope, a gravitysensor, a rotation vector sensor, or a magnetometer) may be used tocollect other types of motion data to generate movement measures.

The advantage of utilizing movement measure is that it is very cheap tooperate on an embedded device. For example, in some embodiments, singlenumeric values of movement measures may capture sufficiently meaningfulinformation to perform the determinations described later herein, butrequire relatively little power to calculate, require relatively littlestorage space to store, require less bandwidth to transmit, etc.Further, a limited number of bits may be used to represent the movementmeasure (e.g., as described above, the movement measure can be cast to a4 bit variable), which minimizes data volume. As discussed above, themovement measure may be transferred off the wearable electronic deviceto a computer server, or an application on an electronic device (an appon a mobile phone or tablet, for example) for further processing, suchas making determinations of the state of the user.

Statistical Measures of User Movement

As previously described with reference to task box 3 shown in FIG. 1,the statistical measurer 122 may perform statistical measures togenerate feature values for a given time. A user's non-instantaneousbehavior can be characterized using distributions (anddistribution-derived features) of movement measures over varioustimescales or window sizes (e.g., 5, 10, 40 minute intervals). Oneembodiment of the statistical measurer 122 uses rolling windows centeredon the moments of interest (the time to which the feature value isassigned). For example, a feature value F_(N) using a W minute windowcentered at time T will describe a characteristic of the distribution ofmovement measures extending from T−W/2 to T+W/2. In other embodiments,the statistical measurer 122 may use rolling windows that are left- orright-aligned to time T in order to consider only data that occurredprevious to time T or after, respectively. The rate in which the featurevalues are calculated can correspond to the rate in which the movementmeasurements are generated, the window size, or any other suitable rate.For example, the next feature (e.g., F_(N+1)) may correspond to timeT+W, T+W/X (where X is any numerical value), T+MMI (where MMI representsthe interval of the movement measurement (e.g., 30 seconds according tosome embodiments), or T+X.

For determinations of the user's state, multiple sets of statisticalfeatures over multiple sets of time windows may be utilized in oneembodiment. Exemplary statistical features are:

-   -   The rolling fraction of movement measures values below a        threshold value (e.g., 0.13 g or any other suitable value)    -   The rolling fraction of movement measures above a threshold        value or between a range of thresholds (e.g., above 0.17 g or        between 0.17 g and 0.20 g) or any other suitable value    -   The rolling fraction of contiguous movement measures that are        above a threshold value (e.g., 0.2 g) or any other suitable        value    -   Various rolling quantiles, averages, standard deviations, and        other summary statistics of these distributions.

Information from Optical Photoplethysmography (PPG)

As illustrated in FIG. 1, at optional task box 1O, optionally aphotoplethysmographic (PPG) sensor 114 may be utilized to generate PPGdata to calculate heart rate (HR), HR variability (HRV), and/orrespiration rate (RR). A PPG sensor typically includes a light sourceand a photodetector. A common light source is a light-emitting diode(LED).

PPG data can use used to calculate a user's heart rate by measuring thetime between peaks or by calculating a dominant frequency in the opticalsignal. For most users, heart rate drops soon after onset of sleep andcontinues dropping over the course of sleep until early in the morning.Heart rate rises when the users wake up or during short disturbancesduring sleep. Measuring heart rate allows us to better identify periodsof sleep. Further, heart rate is a good indicator to separate periods ofsleep from periods of lying still, which could confuse a motion sensorlike an accelerometer. Some users do not see this characteristic drop inheart rate with sleep. Such users can be identified after they wear thewearable electronic device for a few days and a different algorithm canbe used on such users.

The difference of a user's heart rate from his/her average resting heartrate can be used as a personalized measure of how low his/her heart ratehas dropped. The PPG data can also be used to calculate the respirationrate of the user. Respiration rate often shows distinct changes withsleep and can be used as a feature to identify periods of sleep.Respiration can be calculated using a number of techniques:

-   -   Measuring variabilities in the inter-peak intervals between        heart beats    -   Measuring slow variabilities (0.1-0.5 Hz) in the baseline of the        PPG signal    -   Measuring slow periodic signals (0.1-0.5 Hz) in the acceleration        signal

In one embodiment, data representing the heart rate, heart ratevariability, and/or respiration rate of the user of the wearableelectronic device are calculated by the PPG data analyzer 118, and theresulting data (referred to as “analyzed PPG data”) is an additionalinput to user activity classifier 124 and/or multi-class sleep stageclassifier 128, in addition to the statistical features obtained throughstatistical measurer 122.

Classification Into Activity Levels

As previously discussed with reference to task box 4A shown in FIG. 1,the user activity classifier 124 may classify or otherwise label thestatistical features associated with a time period with an activitylevel, such as being classified with an active level or an inactivelevel. In some embodiments, the user activity classifier 124 may includea classification model generated based on running a supervised machinelearning algorithm on a set of sample features with known labels, suchas a feature set associated with an active state (e.g., where the weareris more likely to be awake) or an inactive state (e.g., where the weareris more likely to be asleep).

In one embodiment, the user activity classifier is a decision tree, buta variety of other classifiers are possible, such as, by way of exampleand not limitation:

-   -   Random forests    -   Support vector machine    -   Neural network    -   K-nearest neighbor    -   Naïve Bayes    -   Hidden Markov Model

In addition, user activity classifier may use boosting to combine thesevarious machine learning classifier models to make more accurateclassification. The user activity classifier can, in one embodiment,post-process the classification models to remove noise and improveclassification using binary-opening and binary-closing operations. Avariety of other smoothing operations are possible, including mediansmoothing, weighted smoothing, Kriging, Markov models, etc.

The accuracy of the inferred transition points between user states canbe improved using a variety of methods including edge-detection,changepoint analysis, etc.

As discussed above, the user activity classifier may use featuresderived from data other than motion data. Such non-motion data derivedfeatures that can also be incorporated into this classifier include:

-   -   Heart rate data, such as PPG    -   Skin or ambient temperature measurements or changes in        temperature over time    -   Galvanic skin response    -   Ambient light measurement (e.g., people tend to sleep in dark        environments)    -   Sound detection to detect patterns associated with sleeping,        such as snoring, deep breathing, or relative little amounts of        noise.

In one embodiment, the plurality of user activities include a thirddetected status denoting when a device is not being worn, and the“not-worn” status can be used to improve accuracy. This “not-worn”status can be inferred from the statistical features derived from themotion sensor, PPG data, and/or data from sensors such as temperaturesensor, ambient light sensor, galvanic skin response sensor, capacitivesensor, humidity sensor and sound sensor.

For example, in the motion sensor only case, the “not-worn” state can bedetected when the wearable electronic device remains in a particularorientation and/or is very still for too long, since this isuncharacteristic of human sleeping patterns. In the motion sensor+PPGsensor case, the “not-worn” state can be detected from the opticalproperties of the PPG data possibly in conjunction with a low-movementstatus (as measured using the motion sensor). Sleep onset and egress mayalso be inferred from the timing and duration of the “not-worn” state.Sleep estimates may improve for users over time in response to theirediting of sleep onset and egress estimates (by using this feedback astraining data or as priors when making classification predictions).While the “not-worn” state can be detected using motion sensor+PPGsensor, alternative embodiments that do not require a PPG sensor arediscussed herein below.

Derivation of Awake and Asleep States

As discussed above with reference to task box 4B shown in FIG. 1, thetime block classifier 126 derives from the activity levels over a set ofperiods of time, blocks of time which the user is in one of a pluralityof sleep states. A sleep state can include an awake state and an asleepstate. The blocks of time can span one or more of the periods of timeassigned an activity level. Consecutive blocks of time can havedifferent sleep states.

The time block classifier 126 can convert time periods assigned anactive level or an inactive level into a sleep state based onempirically derived logic. Generally, active states imply a wake stateand inactive states imply a sleep state, but a variety of contrary edgecases are accounted for during this conversion stage. For instance, ashort inactive period (<60 mins) may be classified as “awake” time whenthere is appreciable step activity (>200 steps) immediately precedingand succeeding the inactive period (within 20 minutes). An example ofwhere this may be helpful, is in correctly identifying hour longmeetings where a user is sitting still as awake time and not a nap.

Through the automatic detection of a user's states, the user of thewearable electronic device no longer needs explicit user instructionthat the user plans on transitioning into the asleep state, which can beinconvenient for a user of the wearable device and error prone, as usersoften forget to notify the wearable electronic device of the user'stransition.

Sleep Stage Detection

Referring back to FIG. 1, the statistical features of motion measurescan instead or also be used to classify a user's sleep stages (note thedashed boxes labeled user sleep state detection 190 and sleep stageclassification 192). For instance, a wearable electronic device mayimplement user sleep state detection 190, but either not implement sleepstage classification or do such classification a different way. Asanother example, a wearable electronic device may implement sleep stageclassification 192, but either implement user sleep state detection adifferent way or not at all (e.g., a wearable electronic device maycontain an interface for the user to provide notification that the userplans on transitioning into the asleep state (e.g., through the userpushing a button of or tap on the wearable electronic device), andrespond by enabling sleep stage classification 192 until the user wakesup (which transition back to the awake state may be determined throughnotification from the user or through automatic detection (as describedherein or through a different technique)). Thus, the user sleep statedetection 190 and the sleep stage classification 192 are independent,but may be use together. Referring back to FIG. 1, at task box 5, basedon the sets of statistical features (optionally based on HR/HRV/RR too),multi-class sleep stage classifier 128 may determine the stages of thesleep of the user.

Human sleep patterns can be described in terms of discrete stages ofsleep. At the highest descriptive level, these break down into REM(rapid eye movement), and non-REM sleep stages, the latter includingvarious light and deep sleep stages. Patterns of movement, heart-rate,heart-rate variability (HRV), and respiration change according to sleepstage. These patterns can be detected using data from a motion sensor(such as a three axis accelerometer) along with a photoplethysmographicsensor that allows measurement of heart-rate, HRV, and respiration.

The data collected from the motion sensor and photoplethysmographicsensor includes:

-   -   Measures of user movement (see discussion of movement measures        herein above)    -   Data for calculating heart-rate, heart rate variability, and        respiration (e.g., as determined using data from the        photoplethysmographic sensor in an electronic wearable device)    -   Time-domain or frequency-domain measure of heart-rate        variability.

Statistical features may be derived using movement measures, andadditionally, raw heart-rate data can be used to define a number ofhigher-level features. For example:

-   -   Rolling median or average heart rate on various timescales        (e.g., over the last minute, last 5 minutes, etc.)    -   Rolling quantile of the heart rate distribution as measured on        various timescales    -   Whether heart rate exceeded/went-below some threshold over the        course of some interval.

Additionally, statistical features may be derived using time derivativeof heart-rate. For example:

-   -   Rolling maximum change in heart rate over various timescales    -   Rolling median, average, or some other quartile of the        distribution of heart-rate derivative over various timescales

Similar statistical features can be derived using heart-rate variabilityor respiration. For example:

-   -   Number of times HRV or respiration events exceeded some        threshold over various timescales.    -   Rolling median, average, or some other quartile of the        distribution of HRV or respiration over various timescales.

Heart rate, the derivative of heart rate, heart rate variability, andrespiration may be normalized or standardized appropriately on a peruser basis account for natural variation across users. For example, auser's resting heart rate may be subtracted from their instantaneousheart rate. Demographic information may also be used to renormalizefeatures to account for variation according to demographics.

Then, the statistical features derived may be used to train amulti-class classifier (via supervised machine learning) 128 to classifyperiods of time as a particular sleep stage (e.g., REM, various stage ofnon-REM (NREM) such as light or deep sleep).

One embodiment uses a random forest classifier. Other classificationmethods include neural networks, hidden-Markov models, support vectormachines, k-mean clustering, and decision trees. Transitions betweenstages may also be detected using other change-detection methods, suchas change point analysis, t-tests and Kolmogorov-smirnov statistics.

The resulting classification labels from the previous step can besmoothed to reduce noise. For example, the resulting classificationsfrom a random forest classifier may be processed using an empiricallycalibrated Markov model that specifies the transitions probabilitiesbetween states. Other smoothing operations may includebinary-erosion/dilation, median smoothing, etc.

Note that automatic user state detection and user sleep stage detectionillustrated in FIG. 1 may be performed concurrently in a single system.The user state detection and user sleep stage detection may use the sameor different statistical features to make their own determinations.

Illustrative Examples of User State Detection

FIG. 2A illustrates movement measures collected at different timewindows for moments of interest, as well as determining statisticalfeatures for one of the moments of interest and classifying that momentof interest based on the statistical features, according to oneembodiment of the invention. As used herein, a “moment of interest” mayrefer to a given point in time, such as a period of time which maycorrespond to a movement measure, statistical feature, or state, and assuch, a moment of interest may be used interchangeably with any suitableindication of a single point in time or a range of time periods. Themovement measures are collected for each time interval (with referenceto FIG. 1, reference 116 and task box 2). Based on the movement measuresand optionally PPG data, within a time window, embodiments compute oneor more statistical features for a moment within that window atreference 212 (with reference to FIG. 1, reference 122 and task box 3).A statistical feature may utilize movement measures from multiple timewindows in one embodiment. The movement measure can have discrete valuesrepresented by a predefined number of bits. The statistical featuresF₁-F_(N) determined for the moment of interest are used to classify themoment of interest into one of the plurality of activity levels of theuser, the activity levels include active/inactive at reference 214 (withreference to FIG. 1, reference 124 and task box 4A).

Note while in one embodiment, the computation of one or more statisticalfeature uses movement measures at a consecutive subset of the timeintervals that fall within a time window that includes the moment ofinterest, in an alternate embodiment, movement measures of otherpatterns (e.g., every other time interval) within the subset may beutilized for the computation.

FIG. 2B illustrates the use of the user's activity levels ofactive/inactive to derive awake/asleep stages according to oneembodiment of the invention. As illustrated, the activity levels areillustrated as either active or inactive over time (at each moment ofinterest). From the active/inactive levels, embodiments derivenon-overlapping, consecutive blocks of time during each of which theuser is in one of a plurality of states; where the states include anawake state and an asleep state, where each of the blocks of time spanone or more of the moments of interests, and where consecutive ones ofthe blocks of time have different ones of the plurality of states (withreference to FIG. 1, block 126 and task block 4B). Specifically, FIG. 2Billustrates the following blocks of time are derived: awake 222, asleep224, awake 226, asleep 228, awake 230, and asleep 232; each of whichspans multiple moments of interest. By way of example, a given block oftime may include moments of time that were classified into differentuser states (e.g., asleep 228). The transitions in the user's states arerepresented by the edges of the blocks of time (i.e., when one blockends and the next begins—e.g., the user transitions from the awake stateto the sleep state at the end of the block of time 222/beginning ofblock 224).

FIG. 3A illustrates movement measure snapshots taken from data duringdifferent time spans according to one embodiment of the invention. Themovement measures are represented by 4 bits, thus 16 different valuesstarting from 0 to 15 (with offset of +1). Any values higher or lowerthan the 15 values are clamped to the highest (15) and lowest values(0). The time span is between 10:00 pm to 10:00 am, and the movementmeasures changes dynamically.

FIG. 3B illustrates determination of a statistical feature for momentsof interest using data during different time spans according to oneembodiment of the invention. The feature (Feature 1) is calculated forevery 30 second interval of time using a rolling window. Feature 1 usesmovement measures between 20 minutes in the past and 20 minutes in thefuture. For example, the calculation of Feature 1 at 4:00 am may proceedas follows. The movement measures generated between 3:40 am and 4:20 amare stored in an array. If the movement measures are generated at 30second intervals, there may be 40 minutes×2 values/minute+1 movementmeasures. The “+1” accounts for the moment of interest corresponding to4:00 am. The statistical features that can be determined include themean, standard deviation, 75th quartile (or any other suitablequantile), etc. In this example, the statistical feature is to find thenumber of movement measures greater than a threshold value, such as 12.For each moment of interest, a percentage of a number of movementmeasures within the associated window greater than the threshold value(e.g., in the above example, 12) is recorded as the value of Feature 1at the moment of interest. That is, F₁=(a number of movement measureswith values>than the threshold value (e.g., 12))/total number ofmovement measures of the window.

FIG. 3C illustrates the classifying user active/inactive activity levelsat moments of interest using data during different time spans accordingto one embodiment of the invention. As illustrated, based on statisticalfeatures discussed herein above, it is determined that user is activeuntil 12:00 am (reference 241). The status of the user turns to inactivebetween 12:00 am and 4:00 am (reference 242), after which the status ofthe user is active for a time period (reference 243), and goes back toinactive state (reference 244), and then returns active at reference245. Afterward, the user toggles between inactive (reference 246) andactive (reference 247). With respect to FIG. 3C, the time period of timecovered by active period 241 may, for example, signify that the momentsof interests (not shown) within the active period 241 were classifiedwith active statuses.

FIG. 3D illustrates the deriving of user awake/asleep states using dataduring different time spans according to one embodiment of theinvention. For most moments of interest, the active statuses result inderiving the user state being awake, and inactive statuses result inderiving the user state being asleep. However, from the inactive 242(representing moments of interest(s) classified as inactive), active 243(representing moments of interest(s) classified as active), and inactive242 (representing moments of interest(s) classified as inactive), FIG.3C illustrates that an asleep state is derived (reference 252). That is,the user may be classified as in the active status for some moments ofinterest, the user may still be determined to be in the asleep state.Embodiments may classify a time span with an “active” status as anasleep state if certain criteria are met. For example, if a user has arestless period where active and still moments are interspersed.Embodiments may employ a majority rules decision, using the statuses ofthe individual moments comprising the time-span, in the situation wherethe preceding time span was classified as an asleep state, thesucceeding time span has an inactive status, and the current “active”time-span has a length within some specific range (10-120 minutes).

FIG. 3E illustrates the deriving of user statuses (e.g.,active/inactive/not-worn status) using data during different time spansaccording to one embodiment of the invention where not-worn status isincluded. The not-worn status is at reference 260 (representing momentsof interest (not shown) classified as not-worn), where the embodimentdetermines that the user is not wearing the electronic wearable devicearound 9:00 am.

FIG. 3F illustrates the deriving of user awake/asleep states using dataduring different time spans according to one embodiment where not-wornstatuses are considered. In the embodiment illustrated, the not-wornstatus was derived to be part of the awake state 275 based on anempirically derived logic—around 9 am, it is likely the user is in theawake state.

A number of embodiments may have rules for assigning a block of time asleep state depending on an off-wrist state. For example, consider thecase where the user wakes up and immediately takes off their device totake a shower. The movement measure may remain low during this time, sothis time span would be classified as inactive and ultimately as asleeptime. But, because this span of time is classified as not-worn, and thepreceding state is “asleep”, and the succeeding is “awake,” embodimentscan correct the asleep state by classifying this block of time as awaketime. Conversely, in the middle of a user's sleep, they somehow satisfythe not-worn criteria. In this case, the “not worn” time span ispreceded and succeeded by “asleep” time. In this case, embodiments mayclassify the not-worn period as asleep time since it's unlikely that theuser actually took off the device in the middle of a sleep period.

Illustrative Examples of User Sleep Stage Detection

FIG. 4A illustrates the determining of statistical features at differenttime windows for moments of interest, as well as determining statisticalfeatures for one of the moments of interest and classifying that momentof interest based on the statistical features, according one embodimentof the invention. FIG. 4A is similar to FIG. 3A, and the same or similarreferences indicate elements or components having the same or similarfunctionalities. The difference is that the statistical features may bedifferent from what is utilized to determine user state. The statisticalfeatures F′₁ to F′_(M) determined for the moment of interest are used todetermine the sleep stage for the moments of interest.

FIG. 4B illustrates the result of the determination of sleep stages forthe moments of interest at different time windows according oneembodiment of the invention. In this embodiment, there are three NREMstages. Other embodiments may have different NREM stages.

Flow Diagram for Movement Measure Generation, User State Detection, andUser Sleep Stage Detection

FIG. 5 is a flow diagram illustrating movement measure generation, andoptional user state detection and/or user sleep stage detection,according to one embodiment of the invention. The method 500 may beimplemented in a wearable electronic device, an electronic device((e.g., server (including hardware and software)/tablet/smartphonecontaining an application (referred to herein as an “app”)) coupled tothe wearable electronic device, or distributed between the two (e.g.,references 502 and 504 may be implemented in a wearable electronicdevice, while references 506 and 508 may be implemented in an electronicdevice coupled to the wearable electronic device).

At reference 502, motion data generated by a motion sensor is received.The motion data may in some cases be accelerometer data received from anaccelerometer (e.g., a three axis accelerometer) that represents motionalong three axes. In an alternate embodiment, the motion sensor is agyroscope, a gravity sensor, a rotation vector sensor, a locationdetection device (e.g., a GPS device, or a device capable of measuringmovement using cell-phone or WiFi triangulation), or a magnetometer.While in one embodiment the motion sensor is in a wearable electronicdevice that may be worn on a user's body (e.g., arm, ankle, or chest) orembedded in user's garment, alternative embodiments may have the motionsensor data generated by another external electronic device and receivedby the wearable electronic device. The motion data may include multiplesamples generated during different time intervals, such as a first timeinterval and a second time interval.

At reference 504, the wearable electronic device generates movementmeasures for time intervals (e.g., every 30 seconds) based on acombination of a quantification of the motion data generated during therespective time intervals. For example, in the case of acceleration datareceived from a three axes accelerometer, the wearable electronic devicemay quantify the acceleration data along each of the three axes duringthe first time interval and then quantify the acceleration data alongeach of the three axes during the second time interval. Thus, themultiple samples of acceleration data received from the two differenttime intervals are each quantified into a single numerical numbers. Thedifferent techniques that can be used to quantify multiple samples ofmotion data are described above.

Again, it is to be appreciated that a movement measure for a timeinterval may be a single numeric number. Further, the single numericnumber can be represented by a predefined number of bits (e.g., fourbits). In one embodiment, the number of bits is selected so that it issmall enough for ease of transmission between different modules withinthe wearable electronic device or between the wearable electronic deviceand an electronic device coupled to the wearable electronic device, andlarge enough to represent sufficiently meaningful data for subsequentoperations described herein.

Optionally at reference 506, based on the movement measures, sleepstates may be assigned to blocks of time based on the movement measures.The states can include an awake state and an asleep state. Theconsecutive blocks of time may refer to different sleep states, such asa number of an asleep state for a first block of time and an awake statefor a second block of time. As discussed herein, sleep states may bedetermined based on generating features for given points of time basedon a window of movement measures. The features may then be used toassign activity levels to given points in time. The activity levels arethen used to assign sleep states to the blocks of time.

Optionally at reference 508, for moments of interest, one of a pluralityof sleep stages of the user is determined Each of the moments ofinterest corresponds to one of the time intervals, and the determinationfor each of the moments of interest is based on the movement measuresfrom a subset of the time intervals that fall within a time window thatincludes that moment of interest. The plurality of sleep stages includea rapid eye movement (REM) stage and a plurality of non-REM stages.

The movement measurements and/or the determined blocks of time with thestates and/or the sleep stages may be presented to the user (e.g., on adisplay of the device or another electronic device (e.g., atablet/smartphone/computer which receives the data from the wearableelectronic device, generates the data, or receives the data from anotherelectronic device (e.g., a server))) or stored in the wearableelectronic device for a period in time sufficient to present orcommunicate such data to a secondary device.

Note that the quantification of the distribution of the motion data maybe combined in a variety of ways as discussed herein above.

Flow Diagrams for Detecting User's Periods of Sleep and Sleep Stage

FIG. 6 is a flow diagram illustrating a method 600 for automaticdetection of user's periods of sleep according to one embodiment of theinvention. Method 600 is an exemplary implementation of reference 506 ofFIG. 5. The method 600 may be implemented in a wearable electronicdevice, an electronic device ((e.g., server (including hardware andsoftware)/tablet/smartphone containing an application (referred toherein as an “app”)) coupled to the wearable electronic device, ordistributed between the two (e.g., references 602 and 604 may beimplemented in a wearable electronic device, while references 606 and608 may be implemented in an electronic device coupled to the wearableelectronic device).

Reference 602 illustrates the receipt, for each of a plurality of timeintervals, a movement measure based on a combination of a quantificationof a distribution of motion data along each of three axes for that timeinterval. The motion data, which represents motion along the three axes,is generated by a motion sensor (which may be in a wearable electronicdevice, or provided to the wearable electronic device by anotherelectronic device). The three axes can be orthogonal axes of thewearable electronic device in one embodiment. The motion measure may begenerated by one or more the variety of motion sensors in a various wayssuch as discussed herein above relating to FIG. 1 and FIG. 5.

Reference 604 illustrates the determination, for each of a plurality ofmoments of interest, a set of one or more statistical features of whichat least one characterizes a distribution of the movement measuresdetermined for a subset of the time intervals that fall within a timewindow that includes the moment of interest. A moment of interest mayrefer to a specific time period of interest and may be usedinterchangeably with the phrase “a period of time.” Each of the momentsof interest corresponds to one of the time intervals, and the sets ofstatistical features characterize the user's movements. In someembodiments, the time window is a rolling window that centers,left-aligned, or right-aligned to the moment of interest. By way ofexample, the time window may be of various time scales such as 5, 10, 40minute intervals while the time interval for movement measures is insmaller time scales such as 10, 30, 60 seconds. The subset of the timeintervals selected are consecutive time intervals in one embodiment. Inan alternate embodiment, the subset of the time intervals are selectedin a different pattern such as every other time interval or every twotime intervals. In one embodiment, one or more of the statisticalfeatures may be determined using multiple time windows at different timescales.

Reference 606 illustrates the classification of each of the plurality ofmoments of interest into one of a plurality of statuses (e.g., activitylevels) of the user based on the set of statistical features determinedfor that moment, where the statuses of the user include active andinactive. In one embodiment, the classification is further based on PPGdata generated by a PPG sensor (e.g., in the wearable electronic deviceor another electronic device which communicates the PPG data to thewearable electronic device). For example, the PPG data from a timewindow that includes the moment of interest is utilized to calculate atleast one or more of the user's heart rate data, heart rate variabilitydata, and/or respiration data (i.e., the analyzed PPG data). Reference606 may be implemented in the various ways described above with regardto FIG. 1, reference 124 and task box 4A.

Reference 608 illustrates the derivation, from the statuses of the userat the moments of interest, non-overlapping, consecutive blocks of timeduring each of which the user is in one of a plurality of states. Thestates include an awake state and an asleep state, where each of theblocks of time spans one or more of the moments of interests, andconsecutive ones of the blocks of time have different ones of theplurality of states. Reference 608 may be implemented in the variousways described above with regard to FIG. 1, reference 126 and task box4B.

FIG. 7 is a flow diagram illustrating automatic detection of user'ssleep stages according to one embodiment of the invention. Method 700 isan exemplary implementation of reference 508 of FIG. 5. Method 700 maybe implemented in a wearable electronic device, an electronic device((e.g., server (including hardware and software)/tablet/smartphonecontaining an application (referred to as an app) coupled to thewearable electronic device, or distributed between the two (e.g.,references 702 and 704 may be implemented in a wearable electronicdevice, while reference 706 may be implemented in an electronic devicecoupled to the wearable electronic device).

Reference 702 illustrates the receipt of, for each of a plurality oftime intervals, a movement measure based on a combination of aquantification of a distribution of motion data along each of three axesfor that time interval. In an embodiment that implements both FIG. 6 andFIG. 7, references 602 and 702 may be the same.

Reference 704 illustrates the determination, for each of a plurality ofmoments of interest, a set of one or more statistical features of whichat least one characterizes a distribution of the movement measuresdetermined for a subset of the time intervals that fall within a timewindow that includes the moment of interest. Each of the moments ofinterest corresponds to one of the time intervals, and the sets ofstatistical features characterize the user's movements. Reference 704may be implemented in the various ways described above with regard toFIG. 1, references 122 and task box 3.

Reference 706 illustrates the classifying of each of the plurality ofmoments of interest into one of a plurality of sleep stages of the userbased on the set of statistical features determined for that moment ofinterest. The sleep stages include a rapid eye movement (REM) stage anda plurality of non-REM stages. In one embodiment, the classification isfurther based on photoplethysmography (PPG) data generated by aphotoplethysmographic sensor in the wearable electronic device. Forexample, the PPG data from a time window that includes the moment ofinterest is utilized to calculate at least one of the user's heart ratedata, heart rate variability data, and respiration data. Reference 706may be implemented in the various ways described above with regard toFIG. 1, references 128 and task box 5.

Note sleep stage detection of method 700 may be performed after it isdetermined that a user is in the asleep state, and the determination ofthe asleep state may be accomplished in different ways (e.g., throughmethod 600, through a user interacting with an interface on a wearableelectronic device (e.g., pushing a button or tapping the housing)).

Automatic Power Consumption Change of the Photoplethysmographic/MotionSensor

In embodiments that can automatically detection a user's state and/or auser's sleep stage, the information may be used to change powerconsumption of various sensors such as a photoplethysmographic sensorand a motion sensor. For example, accurate measure of heart ratevariability requires greater temporal precision in the signal comparedto that required to measure heart rate, thus the light source of thephotoplethysmographic sensor can be measured at a higher sampling rateand/or higher power for some fraction of the sleeping period (and/or anawake period) to achieve a better estimate of the heart ratevariability. Similarly, accuracy of the measure of motion along the axesmay be improved with a higher sampling rate, sensitivity, and/or powerlevel, and this may be helpful for determining, for example, the user'ssleep stage.

On the flip side, once it is detected that a user is in the sleep state,embodiments may reduce power consumption by the photoplethysmographicsensor and/or the motion sensor.

FIG. 8A is a flow diagram illustrating automatically reducing powerconsumption of at least one of a photoplethysmographic sensor and amotion sensor according to one embodiment of the invention. The methodmay be implemented by a wearable electronic device that include a set ofsensors including a motion sensor (e.g., an accelerometer) and aphotoplethysmographic sensor, which contains a light source and aphotodetector (e.g., a LED). At reference 802, responsive to a state ofthe user, as tracked by the wearable electronic device, transitioninginto an asleep state, the wearable electronic device decreases powerconsumption of at least one of the photoplethysmographic sensor andmotion sensor. Then optionally at reference 804, after the decrease, thewearable electronic device periodically temporarily increases the powerconsumption of the at least one of the photoplethysmographic sensor andmotion sensor to generate additional data for sleep stage detection. Atreference 806, responsive to the state of the user, as tracked by thewearable electronic device, transitioning out of the asleep state, thewearable electronic device reverses the decrease of power consumption ofthe at least one of the photoplethysmographic sensor and motion sensor.

In one embodiment, the decrease of the power consumption includes one ofa decrease of a sampling rate of the photoplethysmographic sensor, adecrease of sensitivity of the photoplethysmographic sensor, a decreaseof a power level of the light source of the photoplethysmographicsensor, an entry into a low precision state of the motion sensor, adecrease in sensitivity of the motion sensor, and a decrease of asampling rate of the motion sensor. In one embodiment, the motion sensoris the accelerometer.

In one embodiment, the temporary increase of power consumption includesat least one of increase of a sampling rate of the photoplethysmographicsensor, increase of sensitivity of the photoplethysmographic sensor,increase of a power level of the light source, entry into a highprecision state of the motion sensor, increase of sensitivity of themotion sensor, and an increase of a sampling rate of the motion sensor.

FIG. 8B is a flow diagram illustrating automatically increasing powerconsumption of at least one of a photoplethysmographic sensor and amotion sensor according to one embodiment of the invention.

At reference 812, responsive to a state of the user, as tracked by thewearable electronic device, transitioning into an asleep state, thewearable electronic device increases power consumption of at least oneof the photoplethysmographic sensor and motion sensor. As previouslydescribed, the increasing of the power consumption provides additionalPPG data or motion data for sleep stage detection.

At reference 814, responsive to the state of the user, as tracked by thewearable electronic device, transitioning out of the asleep state, thewearable electronic device reverses the increase of power consumption ofthe at least one of the photoplethysmographic sensor and motion sensor.

While in one embodiment the tracking of the state of the user by thewearable electronic device for the methods of FIG. 8A or 8B is performedby the wearable electronic device, in an alternative embodiments thetracking of the state of the user is done with the assistance of one ormore other electronic devices (e.g., server (including hardware andsoftware)/tablet/smartphone containing an application (referred to as anapp) which communicates back to the wearable electronic device in realtime or near real time the user's state; as well as optionally sensorsoutside of the wearable electronic device (e.g., on the chest of theuser, the mattress or bedside table of the user) that provide data tothe server/table/smartphone to assist in detecting transitions in andout of the asleep state). Both of these approaches detect the user'sstate and are considered within the meaning of “as tracked by thewearable electronic device”.

While in one embodiment the power consumption is changed responsive tothe detection of a user's transitions in and out of the asleep state,alternative embodiments instead or in addition make sure powerconsumption changes responsive to detection of transition between one ormore of the sleep stages.

The change of power consumption of the sensors may cause a visiblechange to the wearable electronic device. For example, the decrease ofpower consumption of the photoplethysmographic sensor may cause thelight source to emit less light (decrease its illumination level) whenthe power level of the light source is reduced. Similarly, the increaseof power consumption of the photoplethysmographic sensor may cause thelight source to emit more light (increase its illumination level) whenthe power level of the light source is increased. Thus, the wearableelectronic device may change its illumination level upon transitioninginto and out of asleep state.

Exemplary Devices Implementing Embodiments with Movement Measures,Automatic Detection of User's Sleep Period/Stage, and Automatic PowerConsumption Change

As previously described, while in some embodiments the operations areimplemented in a wearable electronic device, alternative embodimentsdistribute different ones of the operations to different electronicdevices (FIG. 9 illustrates examples of one such distribution). FIG. 9is a block diagrams illustrating the wearable electronic device and anelectronic device implementing operations disclosed according to oneembodiment of the invention. Wearable electronic device (WED) 902includes processor 942, which may be a set of processors. WED 902includes motion sensor 112, which may be a multi-axis accelerometer, agyroscope, a gravity sensor, a rotation vector sensor, or a magnetometeras discussed herein above. WED 902 may also include other sensors 914,which may include photoplethysmographic sensor 114 illustrated in FIG.1, temperature sensor 921, ambient light sensor 922, galvanic skinresponse sensor 923, capacitive sensor 924, humidity sensor 925, andsound sensor 926. WED 902 also contains non-transitory machine readablestorage medium 918, which includes instructions that implement themovement measure generator 116 discussed above. When executed byprocessor 942, the movement measure generator causes wearable electronicdevice 902 to generate movement measures. In one embodiment,non-transitory machine readable storage medium 918 contains powerconsumption adjuster 917, which performs power consumption adjustment onat least one of motion sensor 112 and the PPG sensor 114 as discussedherein above.

In one embodiment, other sensors 914 are not within wearable electronicdevice 902. These sensors may be distributed around the user. Forexample, these sensors may be placed on the chest of the user, themattress or bedside table of the user, while the wearable electronicdevice is worn by the user.

FIG. 9 also includes an electronic device 900 (e.g., server (includinghardware and software)/tablet/smartphone executing an application(referred to as an app). The electronic device 900 may perform thefunctionalities relating to statistical measurer 122, user activityclassifier 124, time block classifier 126, and/or multi-class sleepstage classifier 128, some or all of which are included in the sleeptracking module (STM) 950, which is stored in the non-transitory machinereadable storage medium 948. When executed by processor 952, STM 950causes electronic device 900 to perform the corresponding operationsdiscussed herein above. Electronic device 900 may contain virtualmachines (VMs) 962A to 962R, each of which may execute a softwareinstance of STM 950. Hypervisor 954 may present a virtual operatingplatform for the virtual machines 962A to 962R.

Extensions and Enabled Applications

A variety of applications and extensions are possible around sleep usinga variety of sensors.

Data from Wearable Electronic Device when Placed on or Near the Bed

A wearable electronic device can be used to monitor onset/egress ofsleep as well as sleep stages even when not physically worn by the user.For example, an electronic device placed on the bed, on the pillow, inthe pillow or on the bedside table can be used to measure a user'ssleep. Such a device would use the accelerometer, microphone, ambientlight sensor or video camera to measure a user's movement, heart rate orrespiration. Such a video camera might have the functionality toilluminate the room and capture video in low light environments orinfrared. These features can then be used to infer details of a user'ssleep and sleep stage as described herein.

Similarly, a wall mounted device like an alarm clock could be equippedwith a video camera and microphone which would measure a user's heartrate, respiration rate and movement and be used to infer details aboutsleep. Since a user is asleep, the user would be very stationary andthis provides a good opportunity to estimate heart rate from video. Sucha video camera might have the functionality to illuminate the room andcapture video in low light environments or infrared. Also, such a cameracould capture heart rate, respiration rate and movement of multipleusers simultaneously. Thus, the movement measures and PPG data may begenerated by sensors and data collection device outside of a wearableelectronic device, and these movement measures and PPG data may insteadand/or in addition to be provided through the wearable electronic deviceor through a more direct transmission (e.g., having their own WiFiconnection, Bluetooth connection, or cellular connection) to anelectronic device to infer details of a user's sleep and sleep stage asdescribed herein.

Since a user is stationary during sleep, this provides an opportunityfor such a system to measure more advanced metrics about the user likesmall changes in respiration rate or small changes in inter beatinterval (of the heart). This information can be used to help detect ordiagnose conditions like sleep apnea and atrial fibrillation.

Sleep Apnea Detection

Sleep apnea can be described as the disruption of normal breathingduring sleep. Given the auto-detection of sleep, sleep apnea can bedetected in multiple ways using a variety of combinations of sensors.For example:

-   -   Reduction in blood oxygenation using pulse oximetry (e.g., as        part of a PPG system utilizing PPG data);    -   Disruption of normal audible breathing patterns as measured by        an audio sensor    -   Change in respiration rate as measured using the PPG system.    -   Change in respiration using a strain gauge (e.g., worn around        the chest)    -   Change in respiration using an accelerometer (that measures the        periodic accelerations of respiration)    -   Change in respiration using a video camera that directly or        indirectly observes respiration    -   Change in respiration using a CO2 sensor that detects changes in        expelled Carbon Dioxide.

Multi-User Sleep Tracking

The sleep patterns (e.g., sleep onset/egress, wakefulness, sleep stages)may be tracked for multiple users at a time. For example:

-   -   An image recording device located on a bedside table could        directly detect sleep by observing one or more users' state of        motion, whether they are breathing in a way consistent with        sleeping, by measuring their heart-rate and heart-rate        variability by detecting the change in color of their skin with        each heartbeat, etc.    -   An image recording device could also directly detect the        appearance or lack of rapid eye movement in one or more sleeping        users, thus directly detecting the difference between REM and        non-REM sleep stages.    -   A smart mattress with one or more accelerometers could detect        and separate motions from multiple sleeping users.    -   An audio sensor could detect and disambiguate the respiration or        snoring patterns for multiple users at a time.

Detecting the Cause of Sleep Disruptions

Some sleep disruptions may result as a consequence of physiological(e.g., sleep apnea) and/or environmental effects. These can be detectedand correlated with sleep disruptions, which in turn could be used by auser to identify things or events that degrade his/her sleep. Forexample:

-   -   A temperature sensor (worn on the body, made as part of the bed,        as part of a device sitting on a bedside table, etc.) could        detect changes in temperatures that may lead to sleep        disruptions.    -   An audio sensor could detect sounds that may disrupt or degrade        a user's sleep.    -   An ambient light sensor could detect bright, persistent,        intermittent, etc. lights that may disrupt sleep.    -   A humidity sensor could detect changes in humidity that may lead        to discomfort or degraded/disrupted sleep.    -   An accelerometer could be used to detect motions (e.g., a large        truck driving by, an earthquake, etc.) that may disrupt sleep.

Automatic Detection of when a Wearable Electronic Device is not beingWorn Based on a Motion Sensor

Some embodiments discussed herein may relate to a wearable electronicdevice capable of detecting when the wearable electronic device is notbeing worn. For example, depending on the arrangement between thehousing and possibly a wristband, a wearable electronic device candetect when the wearable electronic device has been placed in one of anumber of orientations that the wearable electronic device is commonlyplaced when not being worn. In some cases, these orientations may bespecified by not-worn profiles. A “not-worn profile,” as used herein,may be data or logic that specifies a pattern of motion data thatindicates when a wearable electronic device is not worn by the user. Ina given embodiment, the pattern of motion data (e.g., accelerometerdata) may reflect the force of gravity detected by an inertia sensor(e.g., an accelerometer) along one or more axes.

While embodiments are described which reference to a three axisaccelerometer oriented in the wearable electronic device a particularway, alternative embodiments may have a different orientation, which mayrequire predictable changes in the techniques described herein, such astransforming the motion data from one coordinate system to another. Tosimplify the discussion of example embodiments, the negative directionof an axis is referred to the inverse of the axis.

By way of example and not limitation, a particular orientation for axesof motion detected by an accelerometer is now described relative to adisplay. Referring to the display of the wearable electronic device,worn on the user's forearm in the same place the display of a wristwatch would be worn, relative to a clock face: the X axis is along theline formed between 12 and 6 o'clock (the positive direction being the12 to 6 direction) and may also be referred to as the top-bottom axis;the Y axis is along a line formed between 9 and 3 o'clock (that is, fromthe user's elbow to wrist if worn on the left hand) (the positivedirection, in some cases, being the 9 to 3 direction) and may also bereferred to as the left-right axis; the Z axis is along a lineperpendicular to the clock face (the positive direction being out thefront of the clock face) and may also be referred to as the back-frontaxis. Thus, in this example, the X-Y axes form a plane that contains thedisplay/clock face and the X-Z axes form a plane that is perpendicularto the user's forearm.

In one embodiment of the wearable electronic device that is to be wornon the user's forearm has a housing that contains the electronicsassociated with the wearable electronic device, one or more buttons forthe user to interact with, and one or more displays accessible/visiblethrough the housing. The wearable electronic device also can include awristband to secure the wearable electronic device to the user'sforearm, according to one embodiment of the invention. As used herein,the term “wristband” may refer to a band that is designed to fully orpartially encircle a person's forearm near the wrist joint. The band maybe continuous, e.g., without any breaks (it may stretch to fit over aperson's hand or have an expanding portion similar to a dresswatchband), or may be discontinuous, e.g., having a clasp or otherconnection allowing the band to be closed similar to a watchband or maybe simply open, e.g., having a C-shape that clasps the wearer's wrist.

Some example orientations are now described in greater detail.

FIGS. 10A-E illustrate orientations that different exemplary wearableelectronic devices may be placed when not being worn according toembodiments, which may be represented by different not-worn profiles.FIGS. 10A-B illustrate a wearable electronic device having a C-shape andintegrated housing and wristband according to one embodiment of theinvention. FIGS. 10C-D illustrate a wearable electronic device having awristband similar to a dress watchband, according to one embodiment ofthe invention. FIG. 10E illustrate a wearable electronic device worn bya user when the user is engaged in various activities according toembodiments. Note each depicted wearable electronic device in this setof figures may include an accelerometer.

FIG. 10A illustrates a wearable electronic device placed on its side ona flat surface when not being worn according to one embodiment of theinvention. When placed in this orientation, the left-right axis (theY-axis) runs substantially parallel to the force of gravity, and theacceleration along the left-right axis due to gravity will consistentlymeet a condition relative to a threshold acceleration expected alongthat axis when the wearable electronic device is in this orientation.While FIG. 10A illustrates the wearable electronic device on one of itssides, the wearable electronic device may also be commonly placed on itsopposite side, such as on side 1004. Whether the wearable electronicdevice rests on the side shown in FIG. 10A or side 1004, gravity runsalong the left-right axis, but in opposite directions. Thus, where thepositive direction of the accelerometer measure from left to right, theacceleration data along the left-right axis may measure 1 g when thewearable electronic device is oriented as shown in FIG. 10A and but maymeasure −1 g when the wearable electronic device is oriented such thatthe wearable electronic device is on side 1004. As the wearableelectronic device may rest on either side, the wearable electronicdevice may compare the detected acceleration data with either twodifferent thresholds (one for each side (e.g., 1 g for the resting sideshown in FIG. 10A and −1 g for the side 1004) or take the absolute valueof the acceleration data from the left-right axis, which is thencompared to a gravitational threshold (e.g., 1 g).

FIG. 10B illustrates a wearable electronic device resting on a face sideon a flat surface when not being worn according to one embodiment. Whenplaced in this orientation, the back-front axis (the Z-axis) runssubstantially parallel to the force of gravity, and the accelerationalong the back-front axis due to gravity meet a condition relative to athreshold acceleration expected along that axis when the wearableelectronic device is in this orientation.

Note in one embodiment that it is very unlikely that the wearableelectronic device will rest on its back when not being worn due to theshape and flexibility of the wristband forming the C-shape. Thus, FIGS.10A-B illustrate the orientations that the wearable electronic device iscommonly placed when not being worn.

FIG. 10C illustrates a wearable electronic device placed on a flatsurface such that the face side of the wearable electronic device isperpendicular to that flat surface when not being worn according to oneembodiment. When placed in this orientation, the top-bottom axis (theX-axis) runs substantially parallel to the force of gravity, and theacceleration along the top-bottom axis due to gravity will consistentlymeet a condition relative to a threshold acceleration expected alongthat axis when the wearable electronic device is in this orientation.

FIG. 10D illustrates a wearable electronic device being placed such thatthe back side is on a flat surface when not being worn according to oneembodiment. When placed in this orientation, the back-front axis (theZ-axis) runs substantially parallel to the force of gravity, and theacceleration is along the back-front axis due to gravity willconsistently meet a condition relative to a threshold accelerationexpected along that axis when the wearable electronic device is in thisorientation. The physical characteristics of the wearable electronicdevice FIG. 10D are such that it may also commonly be placed in the sameorientation illustrated in FIG. 10B. Comparing FIG. 10D to FIG. 10B, theback-front axis runs in opposite directions relative to gravity. Thus,in an embodiment where the positive direction runs from back to frontand the condition used for FIG. 10D is a determination whether theacceleration data is greater than a gravitational threshold, logicallythe condition for FIG. 10B would be a determination whether theacceleration data is less than a gravitational threshold. Of course, inother embodiments, such logic can be accomplished differently, such asby taking the absolute value of the acceleration data and comparing theabsolute value against the gravitational threshold.

In one embodiment, the threshold acceleration in FIGS. 10A-B is based onthe force on a stationary object needed to counteract gravity and anacceptable degree of tilt; the tilt may be in one or more of theorientation the wearable electronic device is commonly placed in whennot being worn and the object on which the wearable electronic device isresting. For example, in some embodiments, the threshold accelerationequals 1 g multiplied by cosine (X°), wherein X° is selected from therange 0-40°, and more specifically 25-35°, and in one particularembodiment it is 30° for at least the orientation in FIG. 10A. Whileembodiments may use the same degree for all of the orientations thewearable electronic device is commonly placed in when not worn,different embodiments may chose different degrees of tilt for differentones of the orientations.

FIGS. 10A-D illustrate a number of common orientations a wearableelectronic device may be placed when not being worn, and that suchorientation depend on characteristics of the wearable electronic device,including one or more of a shape, a flexibility, and a range of motionof a mechanism (e.g., the wristband) used to wear the wearableelectronic device on the user's forearm in one embodiment. In addition,these orientations are a result of physical characteristics of thewearable electronic device that come into direct physical contact withan object (e.g., the flat surface) on which the wearable electronic isplaced in one embodiment. For example, while FIGS. 10A-D illustrate theorientations of a wearable electronic device on a flat surface, somewearable electronic devices may be designed to be placed differentlywhen not being worn, such as in a watch box with a sloped surface. Thus,the common orientations of a wearable electronic device when not beingworn are device specific, and different embodiments would take intoaccount such common orientation(s) for a given device when selecting oneor more axes along which the force of gravity will cause theacceleration along those axes to consistently meet a condition relativeto a threshold acceleration.

In contrast with the common orientations a wearable electronic devicemay be placed when not being worn, FIG. 10E illustrates orientations ofa wearable electronic device when a user is engaged in variousactivities according to one embodiment of the invention. As illustrated,user 800A is playing active sports (e.g., tennis), user 800B is walking,user 800C is running, user 800D is practicing yoga, user 800E issleeping, user 800F is playing leisure sports (e.g., golf), user 800G isriding a bicycle, user 800H is walking her pet, and user 800I, a pet, iswalking along its master. All of the users are wearing a wearableelectronic device 100, and all but the pet has the wearable electronicdevice 100 worn on the forearm. The pet has the wearable electronicdevice worn on the leg. Thus, the wearable electronic device is designedto be worn in a particular way, and the particular way is unlikely toinclude the wearable electronic device remaining in the one or moreorientations the wearable electronic device is commonly placed when notbeing worn, for a threshold amount of time.

Referring back to FIG. 10E, when a user is active such as users800A-800C and 800E-800I, the acceleration measured by the accelerometervaries dynamically over time, and it is easier to differentiate from anon-worn state. However, when a user is near stationary as users 800Dand 800E, the acceleration measured by the accelerometer varies littleand/or infrequently, making it is harder to differentiate when thewearable electronic device is not being worn.

Yet, even when the user is relatively stationary, it may be uncommon forthe user to keep the wearable electronic device in the same/similarorientations that the wearable electronic device would be when not beingworn. For example, assuming wearable electronic device 100 has thesimilar characteristics of the wearable electronic device illustrated inFIGS. 10A-B. When user 800D is sleeping, the wearable electronic device100 is at the side of his body. Thus, for example, when sleeping it isvery unlikely that the wearable electronic device will be in theorientation illustrated in FIG. 10A (on its side) for a long period oftime. Thus, it may be determined that wearable electronic device 100 isin a not-worn state when the acceleration measured along with theleft-right axis meets a condition relative to a threshold accelerationexpected along that axis. For example, the condition may be that theacceleration is over 0.7 g or below −0.7 g along left-right axis for adeterminable time period, such as 5 minutes straight.

The condition for making the determination of not-worn state may bechosen depending on the likelihood that the wearable electronic devicewhen worn will be in the same orientation that the wearable electronicdevice is commonly placed when not being worn. For example, it isuncommon for the wearable electronic device to remain in the orientationin FIG. 10B when not being worn, but less so than the orientation inFIG. 10A (e.g., user 800D may sleep with the wearable electronic deviceface-down for a period of time). Thus, the condition for making thedetermination of not-worn state may be that the acceleration is below−0.7 g along the back-front axis for 30 minutes straight.

Flow Diagrams for Automatic Detection of when a Wearable ElectronicDevice is not being Worn Based on an Accelerometer

FIG. 11 is a flow diagram illustrating the automatic detection of when awearable electronic device is not being worn based on motion data,according to one embodiment. The method 1100 shown in FIG. 11 may beimplemented in a wearable electronic device, or distributed between thewearable electronic device and another electronic device. The electronicdevice may be a server, a tablet, a smartphone (executing an application(referred to as an app)), a desktop, a laptop, a set top box, or anyother computer device or computer system coupled to the wearableelectronic device. In an example, references 1102 and 1108 may beperformed on the WED, reference 1104 may be performed on the WED and/oranother electronic device, and reference 1106 may be performed onanother electronic device.

Operations of the method 1100 are now discussed. At reference 1102, themotion data is obtained via motion sensor of a wearable electronicdevice. As discussed above, the motion data can be accelerometer data(e.g., acceleration data along a single axis or along multiple axes) andthe motion sensor can be an accelerometer (or accelerometers), agyroscope, a gravity sensor, a rotation vector sensor, a locationdetection device (e.g., a GPS device, or a device capable of measuringmovement using cell-phone or WiFi triangulation), or a magnetometer.

At reference 1104, the wearable electronic device may automaticallydetermine a period of time when the wearable electronic device is notbeing worn based on a comparison of the motion data and a not-wornprofile that specifies a pattern of motion data that is indicative ofwhen the wearable electronic device is not worn by the user. In somecases, the not-worn profile may include a threshold value representingthe force of gravity that is expected along an axis while the wearableelectronic device rests in given orientation.

In one embodiment, the spans of time must be of a minimum length of timeand is of a variable length of time, alternative embodiments implementfixed length spans of time. In one embodiment, the wearable electronicdevice is designed to be worn such that the display of the wearableelectronic device is in the same place the display of a wrist watchwould be located, the axis is parallel to the left-right axis of thedisplay, and the orientation that the wearable electronic device iscommonly placed when not being worn has the left-right axissubstantially parallel to the gravitational force of the earth. The onlyacceleration measure used for purposes of making the automaticdeterminations is along one axis (also referred to herein as singleaxis) even though the accelerometer may be capable of measuringacceleration along multiple axes, alternative embodiments may operatedifferently. For example, in certain such alternative embodiments, theacceleration measure used for purposes of making different ones of theautomatic determinations is along different ones of the axes (theacceleration along the axes are considered independently and differentones of the automatically determined spans of time will be based on theacceleration measure along only one of the axes). As another example, incertain such alternative embodiments, the acceleration measure used forpurposes of making at least certain ones of the automatic determinationsis along two or more of the axes (the acceleration along two or more ofthe axes are considered collectively and the automatically determinedspans of time will be based on the acceleration measure along only thetwo or more of the axes—also referred to herein as collective axes).Different embodiments may implement different combinations of one ormore single axis and collective axes; for example: 1) single axis todetect spans of time in a first orientation and collective axes for asecond orientation; and 2) multiple single axis respectively for a firstand second orientation, and collective axes for a third orientation.

Optionally at reference 1106, based on activity levels assigned to thespans of time, the wearable electronic device may assign a user state toblocks of time. Examples of user states include an awake state and anasleep state. In some cases, consecutive ones of the blocks of time havedifferent states. In one embodiment, the spans of time assigned to thenot worn activity state may result in the wearable electronic deviceassigning an awake state to a portion of a block of time that includesthe span of times. However, other embodiments may operate differently.For example, the spans of time assigned to the not worn activity statemay result in the wearable electronic device assigning an asleep stateto a portion of a block of time that includes the span of times.

While in one embodiment reference 1106 may be performed as describedearlier herein (and thus, the automatic detection of when a wearableelectronic device is not being worn based on an accelerometer may beused in conjunction with the sleep tracker described earlier herein),reference 1106 may be performed using other sleep/awake state detectiontechniques.

Optionally at reference 1108, based on the spans of time, adjustmentsare made to one or more of the WED power consumption, storage of sensordata, transmission/receipt of data, and scheduling of firmware upgrades.For example, in one embodiment the WED power consumption is reducedduring at least parts of the spans of time when the wearable electronicdevice is not being worn (e.g., reduce power consumed by one or moresensors by, for example, reducing their sensitivity or completelypowering them down). As another example, in one embodiment the WED usespart of the spans of time to upload and/or download data from anotherelectronic device.

FIG. 12 illustrates exemplary alternative embodiments for implementingblock 1104 from FIG. 11. While exemplary alternative embodiments areillustrated with reference to FIG. 12, it should be understood thatother alternative embodiments are within the scope of the invention.

Reference 1204 illustrates the recordation of each consecutive period oftime as either a worn state or a not-worn state based on theacceleration data measured along an axis of the accelerometer exceedinga threshold acceleration for that period of time. In one embodiment,reference 1204 is performed by the wearable electronic device. While inone embodiment the consecutive periods of time are non-overlapping, inalternative embodiments there may be some overlap. While in oneembodiment of the invention the periods of time are of a fixed length,alternative embodiments may support variable length. While in oneembodiment of the invention the periods of time are 30 seconds,alternative embodiments may select the period of time from within arange, for example, of 5-120 seconds. The one or more samples from theaccelerometer along the axis during a given one of the periods of timeare collectively represented by recording a single state (the worn stateor the not—worn state) for that time period. This reduces data volumefor processing, storage, and/or transmission. As such, the length of theperiods of time are selected to sufficiently reduce data volume whilepreserving sufficiently meaningful data for subsequent operationsdescribed herein. Exemplary embodiments are discussed further hereinwith reference to FIG. 14A.

Further, while in one embodiment the times and states are recorded, inalternative embodiments the times and recorded states are compressed toform the data representing the states (e.g., where the periods of timeare of a fixed length, the data may include a start time followed by astream of 1s and zeros that each represent the state during one of thetime periods; additional compression may also be performed (such as runlength encoding)).

With regard to the above discussed exemplary use of one or more singleaxis and collective axes, in one embodiment separate states are recordedfor each of the one or more single axis and collective axes used. Forexample, in an embodiment that only uses a single axis, there would be asingle stream of recorded states for that axis. In contrast, in anembodiment that used two or more single axis, there would be a separatestream of recorded states for each of those single axis. As anotherexample, in an embodiment that uses one or more collective axes, therewould be a separate stream of recorded states for each of the collectiveaxes.

While in one of the exemplary embodiments flow passes from reference1204 to 1206 (reference 1206 being performed by the WED), in analternative one of the exemplary embodiments flow passes from reference1206 to references 1212-1216 (reference 1212 being performed by the WEDand references 1214-1216 being performed in another electronic device((e.g., server (including hardware and software)/tablet/smartphonecontaining an application (referred to as an app)) coupled to thewearable electronic device).

Reference 1206 illustrates the derivation of spans of time when the WEDis not being worn based on the states recorded for the consecutiveperiods of time. In one embodiment, each of the spans of time includesat least a threshold consecutive number of the consecutive time periodsrecorded as the not-worn state. In one embodiment, reference 1206 isperformed in real time and includes the detection of at least athreshold consecutive number of the consecutive time periods that wererecorded as the not-worn state to derive the beginning of one of thespans of time (allowing reference 1108 to be performed). Exemplaryimplementations of reference 1206 are described further herein withreference to FIGS. 14A and 15A-B. While in one embodiment of theinvention each of the spans of time must be of a minimum length of time(e.g., see FIGS. 15A-B) and is of a variable length of time, alternativeembodiments implement fixed length spans of time.

Reference 1212 represents the transmission to another electronic deviceof data representing the states recorded for the consecutive periods oftime. While in one embodiment the data representing the states includestimes and the recorded states, in alternative embodiments the times andrecorded states are compressed to form the data representing the statesas described above.

Reference 1214 represents the receipt at the other electronic device ofthe data representing the states recorded for the consecutive periods oftime. A variety of techniques may be used for implementing thecommunication represented by references 1212-1214, includingwired/wireless and through one or more networks (including theInternet).

Reference 1215 illustrates the derivation of spans of time when the WEDis not being worn based on the data representing the states recorded forthe consecutive periods of time. Reference 1215 may be performed insimilar fashion to reference 1206, or in a more advanced fashion wherethe other electronic device has more processing power and storage.

Optional reference 1216 illustrates the transmission of indications fromthe other electronic device to the WED upon determinations that the WEDis currently in a span of time when the WED is not being worn. Thetransmissions of reference 1216 are utilized in one embodiment of theinvention by the WED to perform reference 1108.

Operations Relating to Automatic Not-Worn State Detection

FIG. 13 illustrate operations relating to not-worn state detectionutilizing accelerometer measures according to one embodiment of theinvention. The task boxes and blocks of FIG. 13 may be implemented in awearable electronic device, or distributed between the wearableelectronic device and one or more other electronic devices coupled tothe wearable electronic device, where the one or more other electronicdevices may, for example, be an electronic device (e.g., server(including hardware and software)/tablet/smartphone containing anapplication (referred to as an app)) to implement blocks 1322/1350. Taskboxes 1-4 illustrate the order in which operations are performed by thecomponents shown by blocks 1312-1370 according to one embodiment of theinvention. It is to be appreciated that although the foregoing isdiscussed with reference to an accelerometer and acceleration data,other embodiments may utilize other types of motion sensors and motiondata.

At task box 1, accelerometer 1312 generates acceleration data for one ormore axes. For example, the acceleration may be sampled at 20 Hz for anaxis, which is one sample every 0.05 seconds for that axis. Whenaccelerometer 1312 generates the acceleration data only for one axis,the data will be forwarded to first state recorder 1314 and the secondstate recorder 1315 and third state recorder 1316 will not be utilized(or even present). When the acceleration data is generated for multipleaxes, and the acceleration data for the multiple axes is to be used forthe automatic not-worn state detection, one or more other staterecorders (e.g., second recorder 1315 and third state recorder 1316) maybe utilized.

At task box 2, one or more state recorders record each consecutiveperiod of time as either a worn state or a not-worn state based onwhether the acceleration data for a given axis meets for that period oftime meets a condition relative to a threshold acceleration. In oneembodiment, each state recorder operates in similar fashion as describedin relation to reference 1204. That is, each recorder may compare themotion data for a given axis with a not-worn profile. As discussedherein above, one state recorder may record the periods of time based onacceleration data for single axis or collective axes.

As discussed above, the threshold accelerations for different staterecorders may have different values. For example, a thresholdacceleration for a common orientation A for not worn (e.g., the Y-axisas discussed relating to FIG. 10A) may be higher than the thresholdacceleration for another common orientation B (e.g., the X-axis asdiscussed herein relating to FIG. 10C), as the common orientation Aresults in the Y-axis running closer to parallel to the force of gravitythan the common orientation B results in the X-axis running parallel tothe force of gravity for example. In that case, the state recordercorresponding to common orientation A may have a higher thresholdacceleration than the one corresponding to common orientation B.

Then at task box 3, not-worn time span classifier 1322 derives spans oftime when the wearable electronic device is not-worn based on datarepresenting the states recorded for the consecutive periods of time.The various ways to derive the spans of time are discussed herein abovefor example in relation to FIG. 12 references 1206 and 1215. Note thethreshold consecutive number of the consecutive time periods recorded asthe not-worn state for one axis may be different for different ones ofthe state recorders as illustrated in FIGS. 15A-B.

Note that the state recorders and the not-worn time span classifiersutilize various thresholds (e.g., threshold acceleration and thresholdconsecutive number of the consecutive time periods recorded as thenot-worn state), and these threshold may be different for differentcommon orientations of the wearable electronic device when not beingworn, as may be specified by different not-worn profiles.

Optionally at task box 4A, based on the spans of time, sleep tracker1350 determines blocks of time (possibly non-overlapping and/orconsecutive) during each of which the user is in one of a plurality ofstates, where the states include an awake state and an asleep state,where consecutive ones of the blocks of time have different ones of theplurality of states. Task box 4A may be implemented the various waysdescribed in relation to reference 1106.

Also optionally at task box 4B, WED operation adjuster 1370 adjusts oneor more of the WED power consumption, storage of sensor data,transmission/receipt of data, and scheduling of firmware upgrades. Taskbox 4B may be implemented the various ways described in relation toreference 1108.

FIG. 14A illustrates the recordation of consecutive periods of time aseither a worn state or a not-worn state based on acceleration datameasured along an axis of an accelerometer exceeding a thresholdacceleration for that period of time according to one embodiment of anot-worn state. In each time period 1402-1406, a number of accelerationdata samples are generated for the axis, and each acceleration sample iscompared with a threshold acceleration used for recording not-worn state(reference 1420). In this example, a time period is recorded as in anot-worn state when values of all the samples in the time period areover the threshold acceleration. To illustrate, the sampled accelerationat references 1432 and 1436 all exceed the threshold acceleration, and,thus, the time periods 1402 and 1406 are recorded as not-worn periods.In contrast, one value of the acceleration data in time period 1404 isbelow the threshold acceleration, and time period 1404 is recorded asworn state. The implementation of recording a not-worn state only if allthe acceleration data is over the threshold can be used to reduceincorrectly detecting the not-worn state (false alarms).

FIG. 14B illustrates the derivation of spans of time when a wearableelectronic device is not being worn based on the states recorded forconsecutive periods of time according to one embodiment of theinvention. The recorded states for the time periods are illustrated inthe figure, where each time period is recorded as either worn (blackblock) or not-worn (white blocks) state. A time span of not wornincludes consecutive time periods that the wearable electronic device isrecorded as not-worn. As illustrated, time span at reference 1422 endsat the occurrence of a single time period of worn state. Note the spanof time has to be long enough (over a threshold value of consecutivetime periods) for a span of time of not-worn to be detected. If a set ofconsecutive time periods recorded as not-worn is interrupted prior toreaching the threshold, the span of time is derived as a worn state.

The threshold time period may, for example, be chosen from the range 5minutes-2 hours. Also, the threshold time period may be different fordifferent ones of the common orientations: for example, for a first oneof the common orientations (e.g., FIG. 10A) the threshold time is chosenfrom the range 5 minutes to 120 minutes (e.g., 30 minutes), while for asecond one of the common orientations (e.g., FIG. 10B) the range is 10to 180 minutes (e.g., 60 minutes).

FIG. 15A illustrates detection of a span of time being in a not-wornstate for a first axis according to one embodiment of the invention. Inthat embodiment, the span of time being in a not-worn state may not havea fixed length and it extends as long as there is not a worn state timeperiod detected. It is advantageous to be able to detect a span of timewhen the wearable electronic device is not being worn, so thatinformation may be used to for example to adjust operations of thewearable electronic device as discussed herein above. That detection, asillustrated in reference 1512, is based on detection of at least a firstthreshold consecutive number of the consecutive time periods recorded asthe not-worn state for the first axis. The first axis is based on thewearable electronic device being commonly placed in an orientation whennot being worn, and the first threshold consecutive number is based onthe likelihood of the wearable electronic device when worn remaining inthat orientation for a first threshold amount of time.

FIG. 15B illustrates detection of a span of time being in a not-wornstate for a second axis according to one embodiment of the invention.That detection, as illustrated in reference 1522, is based on detectionof at least a second threshold consecutive number of the consecutivetime periods recorded as the not-worn state for the second axis. Thesecond axis is based on the wearable electronic device being commonlyplaced in an orientation when not being worn, and the second thresholdconsecutive number is based on the likelihood of the wearable electronicdevice when worn remaining in that orientation for a second thresholdamount of time. That is, the detection of the span of time being in anot-worn state for different axes may be based on different thresholdsof consecutive number of time periods.

Exemplary Devices Implementing Embodiments of Automatic Detection ofNot-Worn State

As previously described, while in some embodiments the operations areimplemented in a wearable electronic device, alternative embodimentsdistribute different ones of the operations to different electronicdevices (FIG. 16 illustrates examples of one such distribution). FIG. 16is a block diagram illustrating the wearable electronic device and anelectronic device implementing operations disclosed according to oneembodiment of the invention. Wearable electronic device (WED) 1602 andelectronic device 1600 are similar to WED 902 and electronic device 900respectively, and the same or similar references indicate elements orcomponents having the same or similar functionalities.

WED 1602 includes motion sensor(s) 1312 to generate motion data, such asacceleration data. It also has non-transitory machine readable storagemedium 1618, which contains one or more state recorders 1620 asdiscussed herein above. The non-transitory machine readable storagemedium 1618 may also include a not-worn time span classifier 1322 toclassify when the derivation of spans of time being in a not-worn state.Similarly, the non-transitory machine readable storage medium 1618 mayalso include a sleep tracker 1350 and a WED operation adjuster 1370 whenthe associated operations are performed in the WED.

Electronic device 1600 has non-transitory machine readable storagemedium 1648, which optionally contains not-worn time span classifier1622 to classify when the derivation of spans of time being in anot-worn state is performed at electronic device 1600; and it containssleep tracker 1650 when the determining of the user states, includingawake and sleep states, is performed at electronic device 1600. In oneembodiment, the sleep tracker 1650 is the sleep tracking module 950 ofFIG. 9.

When executed by processor 1652, NTSC 1622 causes electronic device 1600to perform the corresponding operations discussed herein above.Electronic device 1600 may contain virtual machines (VMs) 1662A to1662R, each of which may execute a software instance of NTSC 1622.Hypervisor 1654 may present a virtual operating platform for the virtualmachines 1662A to 1662R.

While the flow diagrams in the figures herein above show a particularorder of operations performed by certain embodiments, it should beunderstood that such order is exemplary (e.g., alternative embodimentsmay perform the operations in a different order, combine certainoperations, overlap certain operations, etc.).

While the invention has been described in terms of several embodiments,those skilled in the art will recognize that the invention is notlimited to the embodiments described, can be practiced with modificationand alteration within the spirit and scope of the appended claims. Thedescription is thus to be regarded as illustrative instead of limiting.

ALTERNATIVE EMBODIMENTS

Numerous specific details have been set forth herein. However, it is tobe understood that embodiments may be practiced without these specificdetails. In other instances, well-known circuits, structures andtechniques have not been shown in detail in order not to obscure theunderstanding of this description. It will be appreciated, however, byone skilled in the art that the invention may be practiced without suchspecific details. Those of ordinary skill in the art, with the includeddescriptions, will be able to implement appropriate functionalitywithout undue experimentation.

References in the specification to “one embodiment,” “an embodiment,”“an example embodiment,” etc., indicate that the embodiment describedmay include a particular feature, structure, or characteristic, butevery embodiment may not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to effect such feature, structure, or characteristicin connection with other embodiments whether or not explicitlydescribed.

Bracketed text and blocks with dashed borders (e.g., large dashes, smalldashes, dot-dash, and dots) may be used herein to illustrate optionaloperations that add additional features to embodiments. However, suchnotation should not be taken to mean that these are the only options oroptional operations, and/or that blocks with solid borders are notoptional in certain embodiments.

In this description and following claims, the terms “coupled” and“connected,” along with their derivatives, may be used. It should beunderstood that these terms are not intended as synonyms for each other.“Coupled” is used to indicate that two or more elements, which may ormay not be in direct physical or electrical contact with each other,co-operate or interact with each other. “Connected” is used to indicatethe establishment of communication between two or more elements that arecoupled with each other. A “set,” as used herein refers to any positivewhole number of items including one item.

The operations in the flow diagrams have been described with referenceto the exemplary embodiments of the other figures. However, it should beunderstood that the operations of the flow diagrams can be performed byembodiments other than those discussed with reference to the otherfigures, and the embodiments discussed with reference to these otherfigures can perform operations different than those discussed withreference to the flow diagrams.

The following lists some alternative embodiments, by way of example andnot limitation.

Embodiment 31

An apparatus for automatically detecting periods of sleep of a user of awearable electronic device, the apparatus comprising: a set of one ormore processors; a non-transitory machine readable storage mediumcoupled to the set of one or more processors and having stored thereininstructions, which when executed by the set of one or more processors,cause the apparatus to: obtain a set of features for one or more periodsof time from motion data obtained from a set of one or more motionsensors or data derived therefrom; classify the one or more periods oftime as one of a plurality of statuses of the user based on the set offeatures determined for the one or more periods of time, wherein thestatuses are indicative of relative degree of movement of the user, andderive blocks of time covering the period of time during which the useris in one of a plurality of states, wherein the states include an awakestate and an asleep state.

Embodiment 32

The apparatus of embodiment 31, wherein the classification is performedusing a machine learning classifier.

Embodiment 33

The apparatus of embodiment 31, wherein the plurality of statusesfurther include a not-worn status, during which the user is not wearingthe wearable electronic device.

Embodiment 34

The apparatus of embodiment 31, wherein the apparatus is one of thewearable electronic device and a secondary electronic device is coupledto the wearable electronic device.

Embodiment 35

The apparatus of embodiment 33, wherein the combination of thequantification of the distribution of the motion data comprises acombination of a statistical measure of the motion data along each ofthe three axes.

Embodiment 36

The apparatus of embodiment 31, wherein the classification of each ofthe periods of time is further based on photoplethysmography (PPG) datafrom a time window that includes that period of time, and wherein thePPG data is generated by a photoplethysmographic sensor in the wearableelectronic device.

Embodiment 37

The apparatus of embodiment 36, wherein the PPG data is utilized tocalculate at least one of the user's: heart rate data; heart ratevariability data; and respiration data.

Embodiment 38

The apparatus of embodiment 31, wherein the instructions, when executedby the set of processors, also cause the apparatus further to: after thederivation of the user being in an asleep state, determine, for theperiods of time at which the user is in the asleep state, another set ofone or more statistical features of which at least one characterizes adistribution of movement of the user, and classify each of the periodsof time into a plurality of sleep stages of the user based on the set ofstatistical features determined for that moment of interest, wherein thesleep stages include a rapid eye movement (REM) stage and a plurality ofnon-REM stages.

Embodiment 39

The apparatus of embodiment 31, wherein the classification of each ofthe periods of time is further based on data generated by a set ofadditional sensors in the wearable electronic device, wherein the setincludes one or more of the following: temperature sensor; ambient lightsensor; galvanic skin response sensor; capacitive sensor; humiditysensor; and sound sensor.

Embodiment 40

The apparatus of embodiment 31, wherein the set of motion sensorsinclude an accelerometer.

Embodiment 41

A method comprising: obtaining a set of features for one or more periodsof time from motion data obtained from a set of one or more motionsensors or data derived therefrom; classifying the one or more periodsof time as one of a plurality of statuses of the user based on the setof features determined for the one or more periods of time, wherein thestatuses are indicative of relative degree of movement of the user; andderiving blocks of time covering the period of time during which theuser is in one of a plurality of states, wherein the states include anawake state and an asleep state.

Embodiment 42

The method of embodiment 41, wherein the classification is performedusing a machine learning classifier.

Embodiment 43

The method of embodiment 41, wherein the plurality of statuses furtherinclude a not-worn status, during which the user is not wearing thewearable electronic device.

Embodiment 44

The method of embodiment 41, wherein the apparatus is one of thewearable electronic device and a secondary electronic device is coupledto the wearable electronic device.

Embodiment 45

The method of embodiment 43, wherein the combination of thequantification of the distribution of the motion data comprises acombination of a statistical measure of the motion data along each ofthe three axes.

Embodiment 46

The method of embodiment 41, wherein the classification of each of theperiods of time is further based on photoplethysmography (PPG) data froma time window that includes that period of time, and wherein the PPGdata is generated by a photoplethysmographic sensor in the wearableelectronic device.

Embodiment 47

The method of embodiment 46, wherein the PPG data is utilized tocalculate at least one of the user's: heart rate data; heart ratevariability data; and respiration data.

Embodiment 48

The method of embodiment 41, wherein the instructions, when executed bythe set of processors, also cause the apparatus further to: after thederivation of the user being in an asleep state, determining, for theperiods of time at which the user is in the asleep state, another set ofone or more statistical features of which at least one characterizes adistribution of movement of the user, and classifying each of theperiods of time into a plurality of sleep stages of the user based onthe set of statistical features determined for that moment of interest,wherein the sleep stages include a rapid eye movement (REM) stage and aplurality of non-REM stages.

Embodiment 49

The method of embodiment 41, wherein the classification of each of theperiods of time is further based on data generated by a set ofadditional sensors in the wearable electronic device, wherein the setincludes one or more of the following: temperature sensor; ambient lightsensor; galvanic skin response sensor; capacitive sensor; humiditysensor; and sound sensor.

Embodiment 50

The method of embodiment 41, wherein the set of motion sensors includean accelerometer.

Embodiment 51

A computer readable storage device that includes instructions that, whenexecuted by one or more processors, cause the one or more processors to:obtain a set of features for one or more periods of time from motiondata obtained from a set of one or more motion sensors or data derivedtherefrom; classify the one or more periods of time as one of aplurality of statuses of the user based on the set of featuresdetermined for the one or more periods of time, wherein the statuses areindicative of relative degree of movement of the user; and derive blocksof time covering the period of time during which the user is in one of aplurality of states, wherein the states include an awake state and anasleep state.

Embodiment 52

The computer readable storage device of embodiment 51, wherein theclassification is performed using a machine learning classifier.

Embodiment 53

The computer readable storage device of embodiment 51, wherein theplurality of statuses further include a not-worn status, during whichthe user is not wearing the wearable electronic device.

Embodiment 54

The computer readable storage device of embodiment 51, wherein theapparatus is one of the wearable electronic device and a secondaryelectronic device is coupled to the wearable electronic device.

Embodiment 55

The computer readable storage device of embodiment 53, wherein thecombination of the quantification of the distribution of the motion datacomprises a combination of a statistical measure of the motion dataalong each of the three axes.

Embodiment 56

The computer readable storage device of embodiment 51, wherein theclassification of each of the periods of time is further based onphotoplethysmography (PPG) data from a time window that includes thatperiod of time, and wherein the PPG data is generated by aphotoplethysmographic sensor in the wearable electronic device.

Embodiment 57

The computer readable storage device of embodiment 56, wherein the PPGdata is utilized to calculate at least one of the user's: heart ratedata; heart rate variability data; and respiration data.

Embodiment 58

The computer readable storage device of embodiment 51, wherein theinstructions, when executed by the set of processors, also cause theapparatus further to: after the derivation of the user being in anasleep state, determine, for the periods of time at which the user is inthe asleep state, another set of one or more statistical features ofwhich at least one characterizes a distribution of movement of the user,and classify each of the periods of time into a plurality of sleepstages of the user based on the set of statistical features determinedfor that moment of interest, wherein the sleep stages include a rapideye movement (REM) stage and a plurality of non-REM stages.

Embodiment 59

The computer readable storage device of embodiment 51, wherein theclassification of each of the periods of time is further based on datagenerated by a set of additional sensors in the wearable electronicdevice, wherein the set includes one or more of the following:temperature sensor; ambient light sensor; galvanic skin response sensor;capacitive sensor; humidity sensor; and sound sensor.

Embodiment 60

The computer readable storage device of embodiment 51, wherein the setof motion sensors include an accelerometer.

Embodiment 61

A wearable electronic device to be worn by a user, the wearableelectronic device comprising: a set of sensors that detectsphysiological data or environmental data of the user; a set of one ormore processors coupled to the set of sensors; and a non-transitorymachine readable storage medium coupled to the set of one or moreprocessors and having stored therein instructions, which when executedby the set of one or more processors, cause the wearable electronicdevice to: based on detecting that a state of the user, as tracked bythe wearable electronic device, has transitioned into an asleep state,decrease power consumption of at least one sensor from the set ofsensors, and based on detecting that the state of the user, as trackedby the wearable electronic device, has transitioned out of the asleepstate, reverse the decrease of power consumption of the at least onesensor.

Embodiment 62

The wearable electronic device of embodiment 61, wherein the set ofsensors includes a photoplethysmographic sensor to generatephotoplethysmography (PPG) data of the user, wherein thephotoplethysmographic sensor includes a light source and aphotodetector.

Embodiment 63

The wearable electronic device of embodiment 62, wherein the decrease ofthe power consumption includes at least one of: decrease of a samplingrate of the photoplethysmographic sensor, decrease of sensitivity of thephotoplethysmographic sensor, and decrease of a power level of the lightsource.

Embodiment 64

The wearable electronic device of embodiment 62, wherein theinstructions, when executed by the set of processors, also cause thewearable electronic device to: after the decrease, increasing the powerconsumption of the photoplethysmographic sensor to generate additionalPPG data for sleep stage detection.

Embodiment 65

The wearable electronic device of embodiment 64, wherein the increase ofpower consumption of the photoplethysmographic sensor includes at leastone of: increase of a sampling rate of the photoplethysmographic sensor,increase of sensitivity of the photoplethysmographic sensor, andincrease of a power level of the light source.

Embodiment 66

The wearable electronic device of embodiment 61, wherein the set ofsensors includes a motion sensor to generate motion data of the user.

Embodiment 67

The wearable electronic device of embodiment 66, wherein the decrease ofthe power consumption includes at least one of: entering a low precisionstate of the motion sensor, decrease sensitivity of the motion sensor,and decrease of a sampling rate of the motion sensor.

Embodiment 68

The wearable electronic device of embodiment 66, wherein the motionsensor is an accelerometer.

Embodiment 69

The wearable electronic device of embodiment 66, wherein theinstructions, when executed by the set of processors, also cause thewearable electronic device to: after the decrease and before thereverse, periodically temporarily increase the power consumption of themotion sensor to generate additional motion data for sleep stagedetection.

Embodiment 70

The wearable electronic device of embodiment 69, wherein the increase ofpower consumption of the motion sensor includes at least one of:entering a high precision state of the motion sensor, increase ofsensitivity of the motion sensor, and increase of a sampling rate of themotion sensor.

Embodiment 71

A wearable electronic device to be worn by a user, the wearableelectronic device comprising: a set of sensors including at least oneof: a photoplethysmographic (PPG) sensor to generate PPG data of theuser, wherein the photoplethysmographic sensor includes a light sourceand a photodetector, a motion sensor to generate motion data of theuser; a set of one or more processors coupled to thephotoplethysmographic sensor; a non-transitory machine readable storagemedium coupled to the set of one or more processors and having storedtherein instructions, which when executed by the set of one or moreprocessors, cause the wearable electronic device to: based on motiondata generated by the motion sensor, determine that the user issleeping; responsive to the determination that the user is sleeping,increase power consumption of at least one of the photoplethysmographicsensor and the motion sensor, based on motion data generated by themotion sensor, determine that the user is awake; and responsive to thedetermination that the user is awake, reverse the increase of powerconsumption of the at least one of the photoplethysmographic sensor andthe motion sensor.

Embodiment 72

The wearable electronic device of embodiment 71, wherein the increase inpower consumption generates increases PPG data generated by the PPGsensor, wherein the instructions, which when executed by the set of oneor more processors, also causes the wearable electronic device tocalculate at least one of: a set of heart rates of the user, a set ofheart rate variabilities of the user, and a set of respiration rate ofthe user.

Embodiment 73

The wearable electronic device of embodiment 71, wherein the additionaldata is motion data used to generate movement measures at time intervalsbased on a combination of a quantification of a distribution of themotion data along each of the three axes during that time interval, andwherein each of the movement measures is a single numeric number.

Embodiment 74

The wearable electronic device of embodiment 71, wherein the increase ofpower consumption includes at least one of: increase of a sampling rateof the photoplethysmographic sensor, increase of sensitivity of thephotoplethysmographic sensor, and increase of a power level of the lightsource.

Embodiment 75

The wearable electronic device of embodiment 71, wherein the increase ofpower consumption includes at least one of: entering a high precisionstate of the motion sensor, increase of sensitivity of the motionsensor, and increase of a sampling rate of the motion sensor.

Embodiment 76

The wearable electronic device of embodiment 71, wherein the lightsource is a light-emitting diode (LED).

Embodiment 78

A method of managing power consumption of a wearable electronic device,wherein the wearable electronic device includes a set of sensors, themethod comprising: based on detecting that a state of the user, astracked by the wearable electronic device, has transitioned into anasleep state, decrease power consumption of at least one sensor from theset of sensors; and based on detecting that the state of the user, astracked by the wearable electronic device, has transitioned out of theasleep state, reverse the decrease of power consumption of the at leastone sensor.

Embodiment 79

The wearable electronic device of embodiment 78, wherein the set ofsensors includes a photoplethysmographic sensor to generatephotoplethysmography (PPG) data of the user, wherein thephotoplethysmographic sensor includes a light source and aphotodetector.

Embodiment 80

The wearable electronic device of embodiment 79, wherein the decrease ofthe power consumption includes at least one of: decrease of a samplingrate of the photoplethysmographic sensor, decrease of sensitivity of thephotoplethysmographic sensor, and decrease of a power level of the lightsource.

Embodiment 81

The wearable electronic device of embodiment 79, wherein theinstructions, when executed by the set of processors, also cause thewearable electronic device to: after the decrease and before thereverse, periodically temporarily increase the power consumption of thephotoplethysmographic sensor to generate additional PPG data for sleepstage detection.

Embodiment 82

The wearable electronic device of embodiment 81, wherein the increase ofpower consumption of the photoplethysmographic sensor includes at leastone of: increase of a sampling rate of the photoplethysmographic sensor,increase of sensitivity of the photoplethysmographic sensor, andincrease of a power level of the light source.

Embodiment 83

The wearable electronic device of embodiment 78, wherein the set ofsensors includes a motion sensor to generate motion data of the user.

Embodiment 84

The wearable electronic device of embodiment 83, wherein the decrease ofthe power consumption includes at least one of: entering a low precisionstate of the motion sensor, decrease sensitivity of the motion sensor,and decrease of a sampling rate of the motion sensor.

Embodiment 85

The wearable electronic device of embodiment 83, wherein theinstructions, when executed by the set of processors, also cause thewearable electronic device to: after the decrease and before thereverse, periodically temporarily increase the power consumption of themotion sensor to generate additional motion data for sleep stagedetection.

Embodiment 86

The wearable electronic device of embodiment 85, wherein the increase ofpower consumption of the motion sensor includes at least one of:entering a high precision state of the motion sensor, increase ofsensitivity of the motion sensor, and increase of a sampling rate of themotion sensor.

Embodiment 87

A method of managing power consumption of a wearable electronic device,wherein the wearable electronic device includes a set of sensors,including at least one of a photoplethysmographic sensor to generatephotoplethysmography (PPG) data of the user, the photoplethysmographicsensor including a light source and a photodetector, and a motion sensorto generate motion data of the user, the method comprising: responsiveto a state of the user, as tracked by the wearable electronic device,transitioning into an asleep state, increasing power consumption of theat least one of the photoplethysmographic sensor and the motion sensor,wherein the increase of power consumption provides additional data forsleep stage detection; and responsive to the state of the user, astracked by the wearable electronic device, transitioning out of theasleep state, reversing the increase of power consumption of the atleast one of the photoplethysmographic sensor and the motion sensor.

Embodiment 88

The method of embodiment 85, wherein the additional data is PPG dataused to calculate at least one of: a set of heart rates of the user, aset of heart rate variabilities of the user, and a set of respirationrate of the user.

Embodiment 89

The method of embodiment 85, wherein the additional data is motion dataused to generate movement measures at time intervals based on acombination of a quantification of a distribution of the motion dataalong each of the three axes during that time interval, and wherein eachof the movement measures is a single numeric number.

Embodiment 90

The method of embodiment 85, wherein the increasing the powerconsumption includes at least one of: increase of a sampling rate of thephotoplethysmographic sensor, increase of sensitivity of thephotoplethysmographic sensor, and increase of a power level of the lightsource.

Embodiment 91

A wearable electronic device comprising: a set of one or more motionsensors to generate motion data; a set of one or more processors; and anon-transitory machine readable storage medium coupled to the motionsensor and the set of one or more processors, the non-transitory machinereadable storage medium having stored therein instructions, which whenexecuted by the set of processors, cause the set of processors to:automatically determine a period of time when the wearable electronicdevice is not being worn based on a comparison of the motion data and anot-worn profile that specifies a pattern of motion data that isindicative of when the wearable electronic device is not worn by theuser, and store, in the non-transitory machine readable storage medium,data associating the period of time with a not-worn state.

Embodiment 92

The wearable electronic device of embodiment 91, wherein the motion dataincludes a number of motion data samples, and the instructions, whenexecuted by the set of processors, cause the device to automaticallydetermine that the wearable electronic device is not worn for the periodof time based on causing the set of processors to: determine that anumber of motion data samples failing to meet a motion measurementthreshold is below a threshold number.

Embodiment 93

The apparatus of embodiment 91, wherein the non-transitory machinereadable storage medium also stores additional data associatingadditional periods of time with the not-worn state, the period of timeand the additional periods of time together representing consecutiveperiods of time, and wherein the instructions, when executed, also causethe set of processors to: derive a span of time covering the consecutiveperiods of time based on the data and the additional data associatingconsecutive periods of time with the not-worn state; and store, in thenon-transitory machine readable storage medium, data associating thespan of time with the not worn state.

Embodiment 94

The apparatus of embodiment 93, wherein the derivation of the span oftime includes instructions that, when executed, cause the set ofprocessors to: detect that the consecutive periods of time includes atleast a threshold consecutive number of periods of time from the periodof time and the additional periods of time that were associated with thenot-worn state.

Embodiment 95

The apparatus of embodiment 91, wherein the instructions, when executedby the set of processors, also cause the wearable electronic device to:automatically cause, based on the not-worn state for the period of time,one or more of a reduction in power consumption of the wearableelectronic device, a discontinuation of storage of sensor data from aset of one or more sensors of the wearable electronic device, acommunication of data to another electronic device, and a receipt of afirmware update from another electronic device.

Embodiment 96

The apparatus of embodiment 91, wherein the instructions, when executedby the set of processors, also cause the set of processors toautomatically determine another period of time when the wearableelectronic device is not being worn based on a subsequent comparison ofthe motion data and another not-worn profile that specifies a differentpattern of motion data that is indicative of when the wearableelectronic device is not worn by the user.

Embodiment 97

The apparatus of embodiment 91, wherein the pattern of motion datacharacterizes an orientation of the wearable electronic device.

Embodiment 98

The apparatus of embodiment 97, wherein the pattern of motion datafurther characterizes an acceptable range of motion for the orientation.

Embodiment 99

The apparatus of embodiment 97, wherein the pattern of motion datacharacterizes the orientation based on a threshold force of accelerationalong one or more axes consistent with a force of gravity being appliedalong the one or more axes.

Embodiment 100

The apparatus of embodiment 97, wherein the pattern of motion datacharacterizes the orientation based on a threshold force of accelerationalong one or more axes consistent with a force of gravity being appliedalong the one or more axes.

Embodiment 101

The apparatus of embodiment 100, wherein the threshold force ofacceleration accounts for a determinable degree of tilt by the wearableelectronic device.

Embodiment 102

The apparatus of embodiment 99, wherein the one or more axes representsan axis running across a display.

Embodiment 103

The apparatus of embodiment 99, wherein the one or more axes representsan axis running through a display.

Embodiment 104

The apparatus of embodiment 91, wherein the instructions, when executedby the set of processors, also cause the set of processors to: transmitto another electronic device data representing the association of thenot worn state and the period of time.

Embodiment 105

The apparatus of embodiment 91, wherein the instructions, when executedby the set of processors, also cause the set of processors to:automatically determine a subsequent period of time when the wearableelectronic device is being worn based on a subsequent comparison of themotion data and the not-worn profile that specifies the pattern ofmotion data that is indicative of when the wearable electronic device isnot worn by the user; and store, in the non-transitory machine readablestorage medium, data associating the subsequent period of time with aworn state.

Embodiment 106

A method executed by a set of one or more processors of a wearableelectronic device, the method comprising: obtaining motion datagenerated by a set of one or more motion sensors of the wearableelectronic device; determining a period of time when the wearableelectronic device is not being worn based on a comparison of the motiondata and a not-worn profile that specifies a pattern of motion data thatis indicative of when the wearable electronic device is not worn by theuser; and storing, in a non-transitory machine readable storage mediumof the wearable electronic device, data associating the period of timewith a not-worn state.

Embodiment 107

The method of embodiment 106, wherein the motion data includes a numberof motion data samples, and the determining that the wearable electronicdevice is not worn for the period of time includes: determining that anumber of motion data samples failing to meet a motion measurementthreshold is below a threshold number.

Embodiment 108

The method of embodiment 106, further comprising: storing, in thenon-transitory machine readable storage medium, additional dataassociating additional periods of time with the not-worn state, theperiod of time and the additional periods of time together representingconsecutive periods of time; deriving a span of time covering theconsecutive periods of time based on the data and the additional dataassociating consecutive periods of time with the not-worn state; andstoring, in the non-transitory machine readable storage medium, dataassociating the span of time with the not worn state.

Embodiment 109

The method of embodiment 108, wherein the deriving comprises: detectingthat the consecutive periods of time includes at least a thresholdconsecutive number of periods of time from the period of time and theadditional periods of time that were associated with the not-worn state.

Embodiment 110

The method of embodiment 106, further comprising: causing, based on thenot-worn state for the period of time, one or more of a reduction inpower consumption of the wearable electronic device, a discontinuationof storage of sensor data from a set of one or more sensors of thewearable electronic device, a communication of data to anotherelectronic device, and a receipt of a firmware update from anotherelectronic device.

Embodiment 111

The method of embodiment 106, further comprising: determining anotherperiod of time when the wearable electronic device is not being wornbased on a subsequent comparison of the motion data and another not-wornprofile that specifies a different pattern of motion data that isindicative of when the wearable electronic device is not worn by theuser.

Embodiment 112

The method of embodiment 106, wherein the pattern of motion datacharacterizes an orientation of the wearable electronic device.

Embodiment 113

The method of embodiment 112, wherein the pattern of motion data furthercharacterizes an acceptable range of motion for the orientation.

Embodiment 114

The method of embodiment 112, wherein the pattern of motion datacharacterizes the orientation based on a threshold force of accelerationalong one or more axes consistent with a force of gravity being appliedalong the one or more axes.

Embodiment 115

The method of embodiment 112, wherein the pattern of motion datacharacterizes the orientation based on a threshold force of accelerationalong one or more axes consistent with a force of gravity being appliedalong the one or more axes.

Embodiment 116

The method of embodiment 115, wherein the threshold force ofacceleration accounts for a determinable degree of tilt by the wearableelectronic device.

Embodiment 117

The method of embodiment 114, wherein the one or more axes represents anaxis running across a display.

Embodiment 118

The method of embodiment 114, wherein the one or more axes represents anaxis running through a display.

Embodiment 119

The method of embodiment 106, further comprising: transmitting toanother electronic device data representing the association of the notworn state and the period of time.

Embodiment 120

The method of embodiment 106, further comprising: determining that asubsequent period of time when the wearable electronic device is beingworn based on a subsequent comparison of the motion data and thenot-worn profile that specifies the pattern of motion data that isindicative of when the wearable electronic device is not worn by theuser; and storing, in the non-transitory machine readable storagemedium, data associating the subsequent period of time with a wornstate.

What is claimed is:
 1. An apparatus for automatically detecting periodsof sleep of a user of a wearable electronic device, the apparatuscomprising: one or more processors; and a non-transitory machinereadable storage medium coupled to the one or more processors and havingstored therein instructions, which when executed by the one or moreprocessors, cause the one or more processors to: obtain motion data fromone or more motion sensors in the wearable electronic device, define aplurality of time windows including a first time window and a secondtime window, the first and second time windows overlapping, obtain a setof feature values from the motion data including a first feature valuecorresponding to a first time interval and determined based on themotion data within the first time window and a second feature valuecorresponding to a second time interval and determine based on themotion data within the second time window, the first time windowcomprising the first time interval and the second time interval, thesecond time interval being adjacent to the first time interval, and thesecond time window comprising the second time interval, classify each ofthe time intervals into one of a plurality of statuses of the user basedon the corresponding feature value, wherein the statuses are indicativeof relative degrees of movement of the user, consolidate one or moreconsecutive time intervals into one or more blocks of time based atleast in part on the statuses corresponding to the one or moreconsecutive time intervals, wherein each block of time is indicative ofthe user being in one of an awake state and an asleep state, and detectthe periods of sleep of the user based on the consolidated blocks oftime.
 2. The apparatus of claim 1, wherein the classification isperformed using a machine learning classifier.
 3. The apparatus of claim1, wherein the plurality of statuses further include a not-worn status,during which the user is not wearing the wearable electronic device. 4.The apparatus of claim 1, wherein the apparatus is one of the wearableelectronic device and a secondary electronic device to be coupled to thewearable electronic device.
 5. The apparatus of claim 1, wherein one ofthe feature values is based on a combination of a quantification of adistribution of motion data along each of three axes for one of the timeintervals.
 6. The apparatus of claim 1, wherein the classification ofthe first time intervals is further based on photoplethysmography (PPG)data from the first time window, and wherein the PPG data is generatedby a photoplethysmographic sensor in the wearable electronic device. 7.The apparatus of claim 6, wherein the PPG data is utilized to calculateat least one of the user's: heart rate data; heart rate variabilitydata; and respiration data.
 8. The apparatus of claim 1, wherein theinstructions, when executed by the one or more processors, also causethe one or more processors further to: after the consolidation of atleast one block of time indicating that the user is in the asleep state,determine, for the time intervals in the at least one block of timeindicating that the user is in the asleep state, another set ofstatistical feature values of which at least one characterizes adistribution of movement of the user, and classify each of the timeintervals, during which the user is in the asleep state, into one of aplurality of sleep stages of the user based on the another set ofstatistical feature values, wherein the sleep stages include a rapid eyemovement (REM) stage and a plurality of non-REM stages.
 9. The apparatusof claim 1, wherein the classification of each of the time intervals isfurther based on data generated by one or more additional sensors in thewearable electronic device, wherein the one or more additional sensorsinclude one or more of the following: temperature sensor; ambient lightsensor; galvanic skin response sensor; capacitive sensor; humiditysensor; and sound sensor.
 10. The apparatus of claim 1, wherein the oneor more motion sensors include an accelerometer.
 11. The apparatus ofclaim 1, wherein the non-transitory machine readable storage mediumfurther has stored therein instructions, which when executed by the oneor more processors, cause the one or more processors to communicate thedetected periods of sleep of the user to a display for presentation tothe user.
 12. A method for automatically detecting periods of sleep of auser of a wearable electronic device, the wearable device comprising oneor more processors, the method comprising: obtaining, using the one ormore processors of the wearable device, motion data from one or moremotion sensors in the wearable electronic device; defining, using theone or more processors of the wearable device, a plurality of timewindows including a first time window and a second time window, thefirst and second time windows overlapping; obtaining, using the one ormore processors of the wearable device, a set of feature values from themotion data including a first feature value corresponding to a firsttime interval and determined based on the motion data within the firsttime window and a second feature value corresponding to a second timeinterval and determine based on the motion data within the second timewindow, wherein the first time window comprising the first time intervaland the second time interval, the second time interval being adjacent tothe first time interval, and the second time window comprising thesecond time interval; classifying, using the one or more processors ofthe wearable device, each of the time intervals into one of a pluralityof statuses of the user based on the corresponding feature value,wherein the statuses are indicative of relative degrees of movement ofthe user; consolidating, using the one or more processors of thewearable device, one or more consecutive time intervals into one or moreblocks of time based at least in part on the statuses corresponding tothe one or more consecutive time intervals, wherein each block of timeis indicative of the user being in one of an awake state and an asleepstate; and detecting, using the one or more processors of the wearabledevice, the periods of sleep of the user based on the consolidatedblocks of time.
 13. The method of claim 12, wherein the classifying isperformed using a machine learning classifier.
 14. The method of claim12, wherein the plurality of statuses further include a not-worn status,during which the user is not wearing the wearable electronic device. 15.The method of claim 12, wherein the method is performed by one of thewearable electronic device and a secondary electronic device to becoupled to the wearable electronic device.
 16. The method of claim 12,wherein one of the feature values is based on a combination of aquantification of a distribution of motion data along each of three axesfor the time intervals.
 17. The method of claim 12, wherein theclassifying of the first time intervals is further based onphotoplethysmography (PPG) data from the first time window, and whereinthe PPG data is generated by a photoplethysmographic sensor in thewearable electronic device.
 18. The method of claim 17, wherein the PPGdata is utilized to calculate at least one of the user's: heart ratedata; heart rate variability data; and respiration data.
 19. The methodof claim 12, further comprising: after the consolidation of at least oneblock of time indicating that the user is in the asleep state,determining, using the one or more processor, for the time intervals inthe at least one block of time indicating that the user is in the asleepstate, another set of statistical feature values of which at least onecharacterizes a distribution of movement of the user, and classifying,using the one or more processors, each of the time intervals, duringwhich the user is in the asleep state, into one of a plurality of sleepstages of the user based on the another set of statistical featurevalues, wherein the sleep stages include a rapid eye movement (REM)stage and a plurality of non-REM stages.
 20. The method of claim 12,wherein the classifying of each of the time intervals is further basedon data generated by one or more additional sensors in the wearableelectronic device, wherein the one or more additional sensors includeone or more of the following: temperature sensor; ambient light sensor;galvanic skin response sensor; capacitive sensor; humidity sensor; andsound sensor.
 21. The method of claim 12, wherein the one or more motionsensors include an accelerometer.
 22. A computer readable storage devicethat includes instructions that are for automatically detecting periodsof sleep of a user of a wearable electronic device and that, whenexecuted by one or more processors, cause the one or more processors to:obtain motion data from one or more motion sensors in the wearableelectronic device; define a plurality of time windows including a firsttime window and a second time window, the first and second time windowsoverlapping; obtain a set of feature values from the motion dataincluding a first feature value corresponding to a first time intervaland determined based on the motion data within the first time window anda second feature value corresponding to a second time interval anddetermine based on the motion data within the second time window, thefirst time window comprising the first time interval and the second timeinterval, the second time interval being adjacent to the first timeinterval, and the second time window comprising the second timeinterval; classify each of the time intervals into one of a plurality ofstatuses of the user based on the corresponding feature value, whereinthe statuses are indicative of relative degrees of movement of the user;consolidate one or more consecutive time intervals into one or moreblocks of time based at least in part on the statuses corresponding tothe one or more consecutive time intervals, wherein each block of timeis indicative of the user being in one of an awake state and an asleepstate; and detect the periods of sleep of the user based on theconsolidated blocks of time.
 23. The computer readable storage device ofclaim 22, wherein the classification is performed using a machinelearning classifier.
 24. The computer readable storage device of claim22, wherein the plurality of statuses further include a not-worn status,during which the user is not wearing the wearable electronic device. 25.The computer readable storage device of claim 22, wherein the computerreadable storage device is in one of the wearable electronic device anda secondary electronic device to be coupled to the wearable electronicdevice.
 26. The computer readable storage device of claim 22, whereinone of the feature values is based on a combination of a quantificationof a distribution of motion data along each of three axes for one of thetime intervals.
 27. The computer readable storage device of claim 22,wherein the classification of the first time intervals is further basedon photoplethysmography (PPG) data from the first time window, andwherein the PPG data is generated by a photoplethysmographic sensor inthe wearable electronic device.
 28. The computer readable storage deviceof claim 27, wherein the PPG data is utilized to calculate at least oneof the user's: heart rate data; heart rate variability data; andrespiration data.
 29. The computer readable storage device of claim 22,wherein the instructions, when executed by the one or more processors,also cause the one or more processors further to: after theconsolidation of at least one block of time indicating that the user isin the asleep state, determine, for the time intervals in the at leastone block of time indicating that the user is in the asleep state,another set of statistical feature values of which at least onecharacterizes a distribution of movement of the user, and classify eachof the time intervals, during which the user is in an asleep state, intoone of a plurality of sleep stages of the user based on the another setof statistical feature values, wherein the sleep stages include a rapideye movement (REM) stage and a plurality of non-REM stages.
 30. Thecomputer readable storage device of claim 22, wherein the classificationof each of the time intervals is further based on data generated by oneor more additional sensors in the wearable electronic device, whereinthe one or more additional sensors include one or more of the following:temperature sensor; ambient light sensor; galvanic skin response sensor;capacitive sensor; humidity sensor; and sound sensor.
 31. The computerreadable storage device of claim 22, wherein the one or more motionsensors include an accelerometer.